[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:12:41 dist-test-slave-9gcw krb5kdc[19177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:12:41 dist-test-slave-9gcw krb5kdc[19177](info): set up 2 sockets
Aug 08 02:12:41 dist-test-slave-9gcw krb5kdc[19177](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 02:12:42.983875 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:12:43 dist-test-slave-9gcw krb5kdc[19177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:12:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:12:43.125749 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34937
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:34937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:43.242518 19192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:43.242861 19192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:43.242969 19192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:43.247201 19192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:12:43.247298 19192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:43.247334 19192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:43.247375 19192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:12:43.247412 19192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:12:43.253700 19192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:34937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:43.254931 19192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:43.256040 19192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:43.265969 19198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:43.266268 19197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:43.266054 19192 server_base.cc:1034] running on GCE node
W20260808 02:12:43.265966 19200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:43.267187 19192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:43.268911 19192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:43.270192 19192 hybrid_clock.cc:648] HybridClock initialized: now 1786155163270140 us; error 70 us; skew 500 ppm
Aug 08 02:12:43 dist-test-slave-9gcw krb5kdc[19177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155163, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:43.274008 19192 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:12:43.275436 19192 webserver.cc:460] Webserver started at http://127.18.184.190:41849/ using document root <none> and password file <none>
I20260808 02:12:43.276234 19192 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:43.276343 19192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:43.276604 19192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:43.279685 19192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "ff6426907a7745b3941e15f51b6f4a71"
format_stamp: "Formatted at 2026-08-08 02:12:43 on dist-test-slave-9gcw"
server_key: "ed5ca9aa59ddf3c8526a2dc60d0999b8"
server_key_iv: "ccc6f0bdb9ae3969e8397375052e2e49"
server_key_version: "encryptionkey@0"
I20260808 02:12:43.280438 19192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "ff6426907a7745b3941e15f51b6f4a71"
format_stamp: "Formatted at 2026-08-08 02:12:43 on dist-test-slave-9gcw"
server_key: "ed5ca9aa59ddf3c8526a2dc60d0999b8"
server_key_iv: "ccc6f0bdb9ae3969e8397375052e2e49"
server_key_version: "encryptionkey@0"
I20260808 02:12:43.285827 19192 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:12:43.288365 19207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:43.289745 19192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:12:43.289902 19192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "ff6426907a7745b3941e15f51b6f4a71"
format_stamp: "Formatted at 2026-08-08 02:12:43 on dist-test-slave-9gcw"
server_key: "ed5ca9aa59ddf3c8526a2dc60d0999b8"
server_key_iv: "ccc6f0bdb9ae3969e8397375052e2e49"
server_key_version: "encryptionkey@0"
I20260808 02:12:43.290023 19192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:43.303629 19192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:43.306824 19192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:43.307042 19192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:43.314807 19192 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:34937
I20260808 02:12:43.314836 19264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:34937 every 8 connection(s)
I20260808 02:12:43.315811 19192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:12:43.318655 19265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:43.324008 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 19192
I20260808 02:12:43.324178 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:12:43.324445 19265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71: Bootstrap starting.
I20260808 02:12:43.325623 19170 external_mini_cluster.cc:1428] Setting key c776838073f7d9e2784007ec2723b392
I20260808 02:12:43.327069 19265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:43.327839 19265 log.cc:824] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:43.330015 19265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71: No bootstrap required, opened a new log
I20260808 02:12:43.333837 19265 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6426907a7745b3941e15f51b6f4a71" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34937 } }
I20260808 02:12:43.334088 19265 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:43.334146 19265 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6426907a7745b3941e15f51b6f4a71, State: Initialized, Role: FOLLOWER
I20260808 02:12:43.334607 19265 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6426907a7745b3941e15f51b6f4a71" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34937 } }
I20260808 02:12:43.334769 19265 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:12:43.334861 19265 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:12:43.334986 19265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:43.335942 19265 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6426907a7745b3941e15f51b6f4a71" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34937 } }
I20260808 02:12:43.336313 19265 leader_election.cc:302] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff6426907a7745b3941e15f51b6f4a71; no voters:
I20260808 02:12:43.336678 19265 leader_election.cc:288] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:12:43.336752 19270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:43.336958 19270 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [term 1 LEADER]: Becoming Leader. State: Replica: ff6426907a7745b3941e15f51b6f4a71, State: Running, Role: LEADER
I20260808 02:12:43.337284 19270 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6426907a7745b3941e15f51b6f4a71" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34937 } }
I20260808 02:12:43.337992 19265 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:12:43.340165 19271 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff6426907a7745b3941e15f51b6f4a71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6426907a7745b3941e15f51b6f4a71" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34937 } } }
I20260808 02:12:43.340312 19271 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:43.340205 19272 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff6426907a7745b3941e15f51b6f4a71. Latest consensus state: current_term: 1 leader_uuid: "ff6426907a7745b3941e15f51b6f4a71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff6426907a7745b3941e15f51b6f4a71" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34937 } } }
I20260808 02:12:43.340962 19272 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71 [sys.catalog]: This master's current role is: LEADER
W20260808 02:12:43.343122 19285 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:12:43.343212 19285 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:12:43.343276 19286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:12:43.344233 19286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 02:12:43 dist-test-slave-9gcw krb5kdc[19177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:43.350665 19286 catalog_manager.cc:1347] Generated new cluster ID: bbed890829824c1f8c665ed043372258
I20260808 02:12:43.350730 19286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:12:43.352476 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:43.328487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54854 (local address 127.18.184.190:34937)
0808 02:12:43.330277 (+ 1790us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:43.330297 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:43.330337 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:12:43.333456 (+ 3119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:43.333471 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:43.333763 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:43.334115 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:43.336518 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:43.337824 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:43.339481 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:43.340256 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:43.340415 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:43.345796 (+ 5381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:43.345804 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:43.345820 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:43.345854 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:43.348588 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:43.349189 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:43.349197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:43.349204 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:43.349365 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:43.349797 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:43.349800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:43.349802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:43.350340 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:43.351188 (+ 848us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:43.351328 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 02:12:43.351623 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1632,"thread_start_us":140,"threads_started":1}
I20260808 02:12:43.368512 19286 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:12:43.369727 19286 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:12:43.379885 19286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff6426907a7745b3941e15f51b6f4a71: Generated new TSK 0
I20260808 02:12:43.380686 19286 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:12:43.421281 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34937
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:43.566222 19292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:43.566531 19292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:43.566610 19292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:43.570823 19292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:43.570919 19292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:43.571015 19292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:12:43.576727 19292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34937
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:43.577981 19292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:43.579139 19292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:43.589237 19297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:43.589273 19292 server_base.cc:1034] running on GCE node
W20260808 02:12:43.589274 19300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:43.589450 19298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:43.589851 19292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:43.590534 19292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:43.591722 19292 hybrid_clock.cc:648] HybridClock initialized: now 1786155163591689 us; error 48 us; skew 500 ppm
Aug 08 02:12:43 dist-test-slave-9gcw krb5kdc[19177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155163, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:43.594642 19292 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:12:43.595513 19292 webserver.cc:460] Webserver started at http://127.18.184.129:45373/ using document root <none> and password file <none>
I20260808 02:12:43.596042 19292 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:43.596099 19292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:43.596267 19292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:43.598373 19292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "a471184d870c4f209453085247996452"
format_stamp: "Formatted at 2026-08-08 02:12:43 on dist-test-slave-9gcw"
server_key: "6fff3ae973741e03070017223b314646"
server_key_iv: "1649a9fcee200bfef7b9b34c8e3618d7"
server_key_version: "encryptionkey@0"
I20260808 02:12:43.598879 19292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "a471184d870c4f209453085247996452"
format_stamp: "Formatted at 2026-08-08 02:12:43 on dist-test-slave-9gcw"
server_key: "6fff3ae973741e03070017223b314646"
server_key_iv: "1649a9fcee200bfef7b9b34c8e3618d7"
server_key_version: "encryptionkey@0"
I20260808 02:12:43.602531 19292 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:12:43.604998 19308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:43.606302 19292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 02:12:43.606431 19292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "a471184d870c4f209453085247996452"
format_stamp: "Formatted at 2026-08-08 02:12:43 on dist-test-slave-9gcw"
server_key: "6fff3ae973741e03070017223b314646"
server_key_iv: "1649a9fcee200bfef7b9b34c8e3618d7"
server_key_version: "encryptionkey@0"
I20260808 02:12:43.606542 19292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:43.619109 19292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:43.622336 19292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:43.622547 19292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:43.623171 19292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:43.624029 19292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:43.624079 19292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:43.624120 19292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:43.624178 19292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:43.634057 19292 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:39875
I20260808 02:12:43.634076 19433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:39875 every 8 connection(s)
I20260808 02:12:43.635000 19292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:12:43.640101 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 19292
I20260808 02:12:43.640236 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:12:43.640513 19170 external_mini_cluster.cc:1428] Setting key 45d510c3595e34292d2a3d08111b6c6c
Aug 08 02:12:43 dist-test-slave-9gcw krb5kdc[19177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155163, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:43.650384 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:43.636850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:41123 (local address 127.18.184.190:34937)
0808 02:12:43.637038 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:43.637043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:43.638517 (+ 1474us) server_negotiation.cc:407] Connection header received
0808 02:12:43.639568 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:43.639571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:43.639633 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:43.639761 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:43.641090 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:43.641842 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:43.643023 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:43.643276 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:43.643399 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:43.646406 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:43.646410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:43.646412 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:43.646436 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:43.648009 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:43.648545 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:43.648551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:43.648555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:43.648659 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:43.649030 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:43.649041 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:43.649043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:43.649373 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:43.649492 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:43.650014 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 02:12:43.650165 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:12:43.651226 19436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:43.637344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34937 (local address 127.18.184.129:41123)
0808 02:12:43.638288 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:43.638391 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:43.639294 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:43.639937 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:43.639944 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:43.640393 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:43.640932 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:43.640948 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:43.641993 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:43.641995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:43.642893 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:43.642903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:43.643447 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:43.643451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:43.643623 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:43.644321 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:43.644348 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:43.646210 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:43.648130 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:43.648135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:43.648150 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:43.648431 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:43.648769 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:43.648772 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:43.648773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:43.648926 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:43.649482 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:43.649487 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:43.649749 (+ 262us) client_negotiation.cc:769] Sending connection context
0808 02:12:43.649919 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 02:12:43.650201 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"thread_start_us":607,"threads_started":1}
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:12:43.652552 19434 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34937
I20260808 02:12:43.652781 19434 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:43.653364 19434 heartbeater.cc:511] Master 127.18.184.190:34937 requested a full tablet report, sending...
I20260808 02:12:43.655046 19224 ts_manager.cc:194] Registered new tserver with Master: a471184d870c4f209453085247996452 (127.18.184.129:39875)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
I20260808 02:12:43.656373 19224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:41123
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:12:43.676182 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34937
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:43.791512 19439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:43.791841 19439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:43.791921 19439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:43.796239 19439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:43.796340 19439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:43.796435 19439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:12:43.801972 19439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34937
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:43.803105 19439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:43.804189 19439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:43.815003 19445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:43.815057 19444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:43.815003 19447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:43.815240 19439 server_base.cc:1034] running on GCE node
I20260808 02:12:43.815759 19439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:43.816440 19439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:43.817622 19439 hybrid_clock.cc:648] HybridClock initialized: now 1786155163817597 us; error 51 us; skew 500 ppm
Aug 08 02:12:43 dist-test-slave-9gcw krb5kdc[19177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155163, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:43.820393 19439 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:12:43.821336 19439 webserver.cc:460] Webserver started at http://127.18.184.130:33183/ using document root <none> and password file <none>
I20260808 02:12:43.821909 19439 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:43.822003 19439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:43.822243 19439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:43.824297 19439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "74bdba3ea84647638f85d8eedd998e40"
format_stamp: "Formatted at 2026-08-08 02:12:43 on dist-test-slave-9gcw"
server_key: "892e0d38f151bedd5137859d012d04e7"
server_key_iv: "4629fc0d5526be517a1e6bb867db8c14"
server_key_version: "encryptionkey@0"
I20260808 02:12:43.824898 19439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "74bdba3ea84647638f85d8eedd998e40"
format_stamp: "Formatted at 2026-08-08 02:12:43 on dist-test-slave-9gcw"
server_key: "892e0d38f151bedd5137859d012d04e7"
server_key_iv: "4629fc0d5526be517a1e6bb867db8c14"
server_key_version: "encryptionkey@0"
I20260808 02:12:43.828771 19439 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 02:12:43.831285 19454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:43.832548 19439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:12:43.832688 19439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "74bdba3ea84647638f85d8eedd998e40"
format_stamp: "Formatted at 2026-08-08 02:12:43 on dist-test-slave-9gcw"
server_key: "892e0d38f151bedd5137859d012d04e7"
server_key_iv: "4629fc0d5526be517a1e6bb867db8c14"
server_key_version: "encryptionkey@0"
I20260808 02:12:43.832809 19439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:43.843328 19439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:43.846554 19439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:43.846781 19439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:43.847430 19439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:43.848330 19439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:43.848403 19439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:43.848476 19439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:43.848527 19439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:43.858104 19439 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:37585
I20260808 02:12:43.858134 19576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:37585 every 8 connection(s)
I20260808 02:12:43.859046 19439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:12:43.863962 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 19439
I20260808 02:12:43.864082 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:12:43.864359 19170 external_mini_cluster.cc:1428] Setting key a3042712db7b94f77b1dafb72b072ecd
Aug 08 02:12:43 dist-test-slave-9gcw krb5kdc[19177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155163, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:43.876235 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:43.861360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:38431 (local address 127.18.184.190:34937)
0808 02:12:43.861500 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:43.861510 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:43.862492 (+ 982us) server_negotiation.cc:407] Connection header received
0808 02:12:43.863994 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:43.863996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:43.864068 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:43.864155 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:43.865736 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:43.866403 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:43.868290 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:43.868626 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:43.868832 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:43.871906 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:43.871911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:43.871914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:43.871949 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:43.873781 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:43.874327 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:43.874334 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:43.874337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:43.874441 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:43.874800 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:43.874807 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:43.874811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:43.875070 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:43.875191 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:43.875914 (+ 723us) server_negotiation.cc:299] Negotiation successful
0808 02:12:43.876054 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:12:43.877089 19580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:43.861756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34937 (local address 127.18.184.130:38431)
0808 02:12:43.862315 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:43.862376 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:43.863733 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:43.864472 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:43.864479 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:43.864861 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:43.865501 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:43.865522 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:43.866610 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:43.866614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:43.868137 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:43.868154 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:43.868893 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:43.868896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:43.869055 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:43.869782 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:43.869800 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:43.871702 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:43.873914 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:43.873920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:43.873933 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:43.874204 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:43.874539 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:43.874541 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:43.874543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:43.874682 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:43.875370 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:43.875376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:43.875632 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 02:12:43.875798 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 02:12:43.876077 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":176,"threads_started":1}
I20260808 02:12:43.878489 19578 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34937
I20260808 02:12:43.878736 19578 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:43.879364 19578 heartbeater.cc:511] Master 127.18.184.190:34937 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
I20260808 02:12:43.880689 19224 ts_manager.cc:194] Registered new tserver with Master: 74bdba3ea84647638f85d8eedd998e40 (127.18.184.130:37585)
I20260808 02:12:43.881407 19224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:38431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:12:43.897356 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34937
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:44.016527 19583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:44.016832 19583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:44.016922 19583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:44.021278 19583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:44.021346 19583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:44.021441 19583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:12:44.027047 19583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34937
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:44.028322 19583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:44.029501 19583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:44.040462 19592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:44.040442 19589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:44.040642 19583 server_base.cc:1034] running on GCE node
W20260808 02:12:44.040427 19590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:44.041338 19583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:44.042022 19583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:44.043185 19583 hybrid_clock.cc:648] HybridClock initialized: now 1786155164043148 us; error 52 us; skew 500 ppm
Aug 08 02:12:44 dist-test-slave-9gcw krb5kdc[19177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155164, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:44.046326 19583 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:12:44.047269 19583 webserver.cc:460] Webserver started at http://127.18.184.131:42685/ using document root <none> and password file <none>
I20260808 02:12:44.047940 19583 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:44.048043 19583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:44.048391 19583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:44.050863 19583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "08fc006e403848279244ccee30ecf123"
format_stamp: "Formatted at 2026-08-08 02:12:44 on dist-test-slave-9gcw"
server_key: "134c207c72a156af283d14d14c581b09"
server_key_iv: "3d1b992243e1110df321eb941695efc7"
server_key_version: "encryptionkey@0"
I20260808 02:12:44.051457 19583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "08fc006e403848279244ccee30ecf123"
format_stamp: "Formatted at 2026-08-08 02:12:44 on dist-test-slave-9gcw"
server_key: "134c207c72a156af283d14d14c581b09"
server_key_iv: "3d1b992243e1110df321eb941695efc7"
server_key_version: "encryptionkey@0"
I20260808 02:12:44.055527 19583 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:12:44.058378 19600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:44.059808 19583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:12:44.059969 19583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "08fc006e403848279244ccee30ecf123"
format_stamp: "Formatted at 2026-08-08 02:12:44 on dist-test-slave-9gcw"
server_key: "134c207c72a156af283d14d14c581b09"
server_key_iv: "3d1b992243e1110df321eb941695efc7"
server_key_version: "encryptionkey@0"
I20260808 02:12:44.060111 19583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:44.079831 19583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:44.083448 19583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:44.083701 19583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:44.084357 19583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:44.085319 19583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:44.085387 19583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:44.085465 19583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:44.085510 19583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:44.095184 19583 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:38509
I20260808 02:12:44.095222 19725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:38509 every 8 connection(s)
I20260808 02:12:44.096194 19583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:12:44.105216 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 19583
I20260808 02:12:44.105361 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:12:44.105728 19170 external_mini_cluster.cc:1428] Setting key 39660a56588b7c8502173efb66723123
Aug 08 02:12:44 dist-test-slave-9gcw krb5kdc[19177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155164, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:44.112192 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:44.098257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:41405 (local address 127.18.184.190:34937)
0808 02:12:44.098433 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:44.098439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:44.099244 (+ 805us) server_negotiation.cc:407] Connection header received
0808 02:12:44.101047 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:44.101051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:44.101116 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:44.101261 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:44.102480 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:44.103045 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:44.104211 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:44.104582 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:44.104761 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:44.107945 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:44.107949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:44.107952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:44.107981 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:44.109654 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:44.110376 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:44.110380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:44.110382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:44.110489 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:44.110947 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:44.110952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:44.110954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:44.111173 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:44.111284 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:44.111827 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 02:12:44.111982 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:12:44.113098 19728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:44.098619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34937 (local address 127.18.184.131:41405)
0808 02:12:44.099082 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:44.099127 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:44.100770 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:44.101428 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:44.101434 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:44.101815 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:44.102335 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:44.102349 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:44.103160 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:44.103162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:44.104062 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:44.104072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:44.104743 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:44.104746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:44.104901 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:44.105692 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:44.105716 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:44.107764 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:44.109821 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:44.109826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:44.109839 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:44.110152 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:44.110596 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:44.110599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:44.110601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:44.110839 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:44.111284 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:44.111293 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:44.111569 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 02:12:44.111781 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 02:12:44.112138 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":115,"threads_started":1}
I20260808 02:12:44.114502 19726 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34937
I20260808 02:12:44.114771 19726 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:44.115301 19726 heartbeater.cc:511] Master 127.18.184.190:34937 requested a full tablet report, sending...
I20260808 02:12:44.116499 19224 ts_manager.cc:194] Registered new tserver with Master: 08fc006e403848279244ccee30ecf123 (127.18.184.131:38509)
I20260808 02:12:44.117341 19224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:41405
I20260808 02:12:44.120795 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:12:44 dist-test-slave-9gcw krb5kdc[19177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155164, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:12:44 dist-test-slave-9gcw krb5kdc[19177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155164, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.184.129@KRBTEST.COM
I20260808 02:12:44.152721 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:44.141334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48936 (local address 127.18.184.129:39875)
0808 02:12:44.141687 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:44.141702 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:44.141724 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:12:44.141899 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:44.141905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:44.142095 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:44.142220 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:44.142984 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:44.143785 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:44.144639 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:44.145033 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:44.145123 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:44.148338 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:44.148349 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:44.148364 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:44.148392 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:44.150317 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:44.150927 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:44.150939 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:44.150941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:44.151043 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:44.151424 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:44.151428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:44.151430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:44.151745 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:44.152048 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:44.152304 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 02:12:44.152490 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":147,"threads_started":1}
W20260808 02:12:44.153941 19339 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48936
Aug 08 02:12:44 dist-test-slave-9gcw krb5kdc[19177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:12:44 dist-test-slave-9gcw krb5kdc[19177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.129@KRBTEST.COM
I20260808 02:12:44.183276 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:44.174550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48944 (local address 127.18.184.129:39875)
0808 02:12:44.174732 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:44.174737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:44.174884 (+ 147us) server_negotiation.cc:407] Connection header received
0808 02:12:44.175069 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:44.175072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:44.175115 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:44.175202 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:44.176013 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:44.176537 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:44.177480 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:44.177721 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:44.177840 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:44.180380 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:44.180383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:44.180386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:44.180411 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:44.181804 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:44.182251 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:44.182254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:44.182255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:44.182332 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:44.182564 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:44.182566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:44.182568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:44.182724 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:44.182843 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:44.183045 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 02:12:44.183148 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 02:12:44.184335 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 19292
I20260808 02:12:44.192705 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 19439
I20260808 02:12:44.202069 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 19583
I20260808 02:12:44.210350 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 19192
2026-08-08T02:12:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2366 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:12:44 dist-test-slave-9gcw krb5kdc[19740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:12:44 dist-test-slave-9gcw krb5kdc[19740](info): set up 2 sockets
Aug 08 02:12:44 dist-test-slave-9gcw krb5kdc[19740](info): commencing operation
krb5kdc: starting...
W20260808 02:12:45.342803 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:12:45 dist-test-slave-9gcw krb5kdc[19740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:12:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:12:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:12:45.480386 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:41343
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:37963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:41343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:45.597530 19758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:45.597827 19758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:45.597869 19758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:45.601897 19758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:12:45.601954 19758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:45.601974 19758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:45.601991 19758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:12:45.602006 19758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:12:45.607458 19758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:37963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:41343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:41343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:45.608547 19758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:45.609582 19758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:45.618844 19765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:45.618880 19767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:45.618911 19758 server_base.cc:1034] running on GCE node
W20260808 02:12:45.619004 19763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:45.619618 19758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:45.620810 19758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:45.622025 19758 hybrid_clock.cc:648] HybridClock initialized: now 1786155165621985 us; error 55 us; skew 500 ppm
Aug 08 02:12:45 dist-test-slave-9gcw krb5kdc[19740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155165, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:45.625434 19758 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:12:45.626616 19758 webserver.cc:460] Webserver started at http://127.18.184.190:35085/ using document root <none> and password file <none>
I20260808 02:12:45.627254 19758 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:45.627355 19758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:45.627621 19758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:45.630079 19758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "47ed9929f11a49e7b7590c03b0d8be07"
format_stamp: "Formatted at 2026-08-08 02:12:45 on dist-test-slave-9gcw"
server_key: "99013a976398f63dd76cb0cf4d3fac69"
server_key_iv: "8f4135865a876dbab10d742c35c5e6fc"
server_key_version: "encryptionkey@0"
I20260808 02:12:45.630815 19758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "47ed9929f11a49e7b7590c03b0d8be07"
format_stamp: "Formatted at 2026-08-08 02:12:45 on dist-test-slave-9gcw"
server_key: "99013a976398f63dd76cb0cf4d3fac69"
server_key_iv: "8f4135865a876dbab10d742c35c5e6fc"
server_key_version: "encryptionkey@0"
I20260808 02:12:45.635372 19758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:12:45.638432 19774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:45.639979 19758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:12:45.640213 19758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "47ed9929f11a49e7b7590c03b0d8be07"
format_stamp: "Formatted at 2026-08-08 02:12:45 on dist-test-slave-9gcw"
server_key: "99013a976398f63dd76cb0cf4d3fac69"
server_key_iv: "8f4135865a876dbab10d742c35c5e6fc"
server_key_version: "encryptionkey@0"
I20260808 02:12:45.640368 19758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:45.659855 19758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:45.663663 19758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:45.663931 19758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:45.673722 19758 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:41343
I20260808 02:12:45.673882 19830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:41343 every 8 connection(s)
I20260808 02:12:45.674875 19758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:12:45.678419 19831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:45.678489 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 19758
I20260808 02:12:45.678607 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:12:45.678910 19170 external_mini_cluster.cc:1428] Setting key b32b10bd49b2dc17fd469ae567158643
I20260808 02:12:45.686678 19831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07: Bootstrap starting.
Aug 08 02:12:45 dist-test-slave-9gcw krb5kdc[19740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:45.689874 19831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:45.690869 19831 log.cc:824] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:45.693284 19831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07: No bootstrap required, opened a new log
I20260808 02:12:45.696017 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:45.680522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59064 (local address 127.18.184.190:41343)
0808 02:12:45.681010 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:45.681027 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:45.681068 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:12:45.682093 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:45.682114 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:45.682513 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:45.682886 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:45.684030 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:45.685339 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:45.686630 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:45.687206 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:45.687336 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:45.690505 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:45.690512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:45.690531 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:45.690567 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:45.692684 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:45.693294 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:45.693300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:45.693304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:45.693509 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:45.693850 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:45.693853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:45.693854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:45.694336 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:45.694497 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:45.694816 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 02:12:45.695181 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":167,"threads_started":1}
I20260808 02:12:45.697609 19831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ed9929f11a49e7b7590c03b0d8be07" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41343 } }
I20260808 02:12:45.697889 19831 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:45.697973 19831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47ed9929f11a49e7b7590c03b0d8be07, State: Initialized, Role: FOLLOWER
I20260808 02:12:45.698410 19831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ed9929f11a49e7b7590c03b0d8be07" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41343 } }
I20260808 02:12:45.698575 19831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:12:45.698647 19831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:12:45.698747 19831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:45.699713 19831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ed9929f11a49e7b7590c03b0d8be07" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41343 } }
I20260808 02:12:45.700073 19831 leader_election.cc:302] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47ed9929f11a49e7b7590c03b0d8be07; no voters:
I20260808 02:12:45.700448 19831 leader_election.cc:288] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:12:45.700558 19837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:45.700866 19837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [term 1 LEADER]: Becoming Leader. State: Replica: 47ed9929f11a49e7b7590c03b0d8be07, State: Running, Role: LEADER
I20260808 02:12:45.701359 19837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ed9929f11a49e7b7590c03b0d8be07" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41343 } }
I20260808 02:12:45.701812 19831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:12:45.703150 19839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47ed9929f11a49e7b7590c03b0d8be07. Latest consensus state: current_term: 1 leader_uuid: "47ed9929f11a49e7b7590c03b0d8be07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ed9929f11a49e7b7590c03b0d8be07" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41343 } } }
I20260808 02:12:45.703308 19839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:45.703652 19838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47ed9929f11a49e7b7590c03b0d8be07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ed9929f11a49e7b7590c03b0d8be07" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41343 } } }
I20260808 02:12:45.703754 19838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07 [sys.catalog]: This master's current role is: LEADER
W20260808 02:12:45.706851 19852 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:12:45.706928 19852 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:12:45.707016 19847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:12:45.707765 19847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:12:45.713799 19847 catalog_manager.cc:1347] Generated new cluster ID: f6c09ef62f7441eba18c4acd88230a35
I20260808 02:12:45.713941 19847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:12:45.723251 19847 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:12:45.724367 19847 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:12:45.729096 19847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47ed9929f11a49e7b7590c03b0d8be07: Generated new TSK 0
I20260808 02:12:45.729949 19847 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:12:45.770615 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:41343
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:37963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:45.902926 19860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:45.903301 19860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:45.903389 19860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:45.908226 19860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:45.908296 19860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:45.908396 19860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:12:45.914623 19860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:37963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:41343
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:45.916091 19860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:45.917383 19860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:45.928594 19866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:45.928575 19868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:45.928862 19860 server_base.cc:1034] running on GCE node
W20260808 02:12:45.928527 19865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:45.929388 19860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:45.930073 19860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:45.931293 19860 hybrid_clock.cc:648] HybridClock initialized: now 1786155165931276 us; error 37 us; skew 500 ppm
Aug 08 02:12:45 dist-test-slave-9gcw krb5kdc[19740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155165, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:45.934614 19860 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:12:45.935683 19860 webserver.cc:460] Webserver started at http://127.18.184.129:40237/ using document root <none> and password file <none>
I20260808 02:12:45.936395 19860 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:45.936463 19860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:45.936697 19860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:45.938948 19860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "67585de9427e478c9f4995217ba42bf4"
format_stamp: "Formatted at 2026-08-08 02:12:45 on dist-test-slave-9gcw"
server_key: "b1094a61189a56037b1727a9d8623df5"
server_key_iv: "ab9899e2eb7c23d62fab5d007de8a06c"
server_key_version: "encryptionkey@0"
I20260808 02:12:45.939555 19860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "67585de9427e478c9f4995217ba42bf4"
format_stamp: "Formatted at 2026-08-08 02:12:45 on dist-test-slave-9gcw"
server_key: "b1094a61189a56037b1727a9d8623df5"
server_key_iv: "ab9899e2eb7c23d62fab5d007de8a06c"
server_key_version: "encryptionkey@0"
I20260808 02:12:45.943585 19860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 02:12:45.946437 19875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:45.947825 19860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:12:45.947978 19860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "67585de9427e478c9f4995217ba42bf4"
format_stamp: "Formatted at 2026-08-08 02:12:45 on dist-test-slave-9gcw"
server_key: "b1094a61189a56037b1727a9d8623df5"
server_key_iv: "ab9899e2eb7c23d62fab5d007de8a06c"
server_key_version: "encryptionkey@0"
I20260808 02:12:45.948120 19860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:45.989287 19860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:45.992964 19860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:45.993180 19860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:45.993839 19860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:45.994663 19860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:45.994719 19860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:45.994760 19860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:45.994774 19860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:46.004873 19860 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:46825
I20260808 02:12:46.004892 19997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:46825 every 8 connection(s)
I20260808 02:12:46.005929 19860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:12:46.009620 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 19860
I20260808 02:12:46.009737 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:12:46.009979 19170 external_mini_cluster.cc:1428] Setting key 9b23604b32b07c29513d0d83f24817df
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[19740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155165, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:12:46.021498 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.007884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:59371 (local address 127.18.184.190:41343)
0808 02:12:46.008048 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:46.008054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:46.009005 (+ 951us) server_negotiation.cc:407] Connection header received
0808 02:12:46.010100 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:46.010104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:46.010162 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:46.010261 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:46.011501 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.012268 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.013582 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.013866 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.013992 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.017221 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:46.017225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:46.017227 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:46.017255 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:46.019037 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.019668 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.019684 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.019686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.019771 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.020180 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.020184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.020186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.020392 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:46.020524 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:46.021119 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 02:12:46.021313 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:12:46.022388 20000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.008329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:41343 (local address 127.18.184.129:59371)
0808 02:12:46.008830 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:46.008880 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:46.009875 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:46.010427 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:46.010434 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:46.010878 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:46.011373 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.011387 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.012404 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.012407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.013452 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.013463 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.014033 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.014038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.014218 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.014857 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:46.014874 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:46.016969 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:46.019227 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:46.019234 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:46.019259 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:46.019538 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:46.019883 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:46.019887 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:46.019889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:46.020073 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:46.020543 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:46.020551 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:46.020842 (+ 291us) client_negotiation.cc:769] Sending connection context
0808 02:12:46.021065 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 02:12:46.021492 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
I20260808 02:12:46.024044 19998 heartbeater.cc:347] Connected to a master server at 127.18.184.190:41343
I20260808 02:12:46.024386 19998 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:46.025177 19998 heartbeater.cc:511] Master 127.18.184.190:41343 requested a full tablet report, sending...
I20260808 02:12:46.026895 19793 ts_manager.cc:194] Registered new tserver with Master: 67585de9427e478c9f4995217ba42bf4 (127.18.184.129:46825)
I20260808 02:12:46.028172 19793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:59371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:12:46.045122 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:41343
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:37963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:46.164093 20005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:46.164402 20005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:46.164448 20005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:46.168912 20005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:46.168975 20005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:46.169075 20005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:12:46.174510 20005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:37963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:41343
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:46.175627 20005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:46.176702 20005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:12:46.187465 20005 server_base.cc:1034] running on GCE node
W20260808 02:12:46.187450 20011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:46.187450 20013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:46.187718 20010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:46.188256 20005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:46.189059 20005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:46.190241 20005 hybrid_clock.cc:648] HybridClock initialized: now 1786155166190212 us; error 41 us; skew 500 ppm
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[19740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155166, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:46.193377 20005 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:12:46.194268 20005 webserver.cc:460] Webserver started at http://127.18.184.130:35487/ using document root <none> and password file <none>
I20260808 02:12:46.194828 20005 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:46.194890 20005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:46.195075 20005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:46.197302 20005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "320511051fc6496497eb7d430d7b5cbd"
format_stamp: "Formatted at 2026-08-08 02:12:46 on dist-test-slave-9gcw"
server_key: "925b55827617c6f7c5f437cc4290db44"
server_key_iv: "45f82a812a58b7e7fec15b0c68ae8d98"
server_key_version: "encryptionkey@0"
I20260808 02:12:46.197918 20005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "320511051fc6496497eb7d430d7b5cbd"
format_stamp: "Formatted at 2026-08-08 02:12:46 on dist-test-slave-9gcw"
server_key: "925b55827617c6f7c5f437cc4290db44"
server_key_iv: "45f82a812a58b7e7fec15b0c68ae8d98"
server_key_version: "encryptionkey@0"
I20260808 02:12:46.201781 20005 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:12:46.204628 20020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:46.205922 20005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:12:46.206058 20005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "320511051fc6496497eb7d430d7b5cbd"
format_stamp: "Formatted at 2026-08-08 02:12:46 on dist-test-slave-9gcw"
server_key: "925b55827617c6f7c5f437cc4290db44"
server_key_iv: "45f82a812a58b7e7fec15b0c68ae8d98"
server_key_version: "encryptionkey@0"
I20260808 02:12:46.206174 20005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:46.232504 20005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:46.235683 20005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:46.235903 20005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:46.236536 20005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:46.237465 20005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:46.237514 20005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:46.237581 20005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:46.237617 20005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:46.247011 20005 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:38019
I20260808 02:12:46.247052 20142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:38019 every 8 connection(s)
I20260808 02:12:46.247972 20005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:12:46.253810 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 20005
I20260808 02:12:46.253950 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:12:46.254254 19170 external_mini_cluster.cc:1428] Setting key b8717fa85c3decddefde1de668baf16e
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[19740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155166, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:46.266101 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.250144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:49403 (local address 127.18.184.190:41343)
0808 02:12:46.250330 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:46.250334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:46.251433 (+ 1099us) server_negotiation.cc:407] Connection header received
0808 02:12:46.252564 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:46.252568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:46.252666 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:46.252766 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:46.254891 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.255727 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.257786 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.258058 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.258160 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.261574 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:46.261579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:46.261583 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:46.261618 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:46.263500 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.264133 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.264140 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.264143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.264245 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.264625 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.264630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.264632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.264914 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:46.265024 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:46.265711 (+ 687us) server_negotiation.cc:299] Negotiation successful
0808 02:12:46.265878 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:12:46.266989 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.250605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:41343 (local address 127.18.184.130:49403)
0808 02:12:46.251243 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:46.251300 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:46.252315 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:46.253043 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:46.253051 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:46.253665 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:46.254465 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.254490 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.255910 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.255913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.257615 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.257633 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.258256 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.258263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.258532 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.259342 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:46.259362 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:46.261352 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:46.263648 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:46.263654 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:46.263669 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:46.263982 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:46.264351 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:46.264353 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:46.264355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:46.264509 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:46.265053 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:46.265059 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:46.265426 (+ 367us) client_negotiation.cc:769] Sending connection context
0808 02:12:46.265599 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 02:12:46.265902 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":181,"threads_started":1}
I20260808 02:12:46.268419 20143 heartbeater.cc:347] Connected to a master server at 127.18.184.190:41343
I20260808 02:12:46.268792 20143 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:46.269474 20143 heartbeater.cc:511] Master 127.18.184.190:41343 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
I20260808 02:12:46.270828 19793 ts_manager.cc:194] Registered new tserver with Master: 320511051fc6496497eb7d430d7b5cbd (127.18.184.130:38019)
I20260808 02:12:46.271696 19793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:49403
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:12:46.289110 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:41343
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:37963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:46.409359 20149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:46.409642 20149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:46.409746 20149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:46.414134 20149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:46.414207 20149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:46.414305 20149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:12:46.419956 20149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:37963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:41343
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:46.421060 20149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:46.422159 20149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:46.432511 20158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:46.432515 20156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:46.432562 20149 server_base.cc:1034] running on GCE node
W20260808 02:12:46.432511 20155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:46.433403 20149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:46.434042 20149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:46.435218 20149 hybrid_clock.cc:648] HybridClock initialized: now 1786155166435198 us; error 36 us; skew 500 ppm
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[19740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155166, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:46.438367 20149 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:12:46.439388 20149 webserver.cc:460] Webserver started at http://127.18.184.131:34047/ using document root <none> and password file <none>
I20260808 02:12:46.439972 20149 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:46.440034 20149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:46.440266 20149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:46.442510 20149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441"
format_stamp: "Formatted at 2026-08-08 02:12:46 on dist-test-slave-9gcw"
server_key: "b70b1c78c9029383598c4949d7b77505"
server_key_iv: "54c4c3b878e745cb98f510fcbcf3b05e"
server_key_version: "encryptionkey@0"
I20260808 02:12:46.443079 20149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441"
format_stamp: "Formatted at 2026-08-08 02:12:46 on dist-test-slave-9gcw"
server_key: "b70b1c78c9029383598c4949d7b77505"
server_key_iv: "54c4c3b878e745cb98f510fcbcf3b05e"
server_key_version: "encryptionkey@0"
I20260808 02:12:46.447194 20149 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:12:46.449934 20166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:46.451514 20149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:12:46.451622 20149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441"
format_stamp: "Formatted at 2026-08-08 02:12:46 on dist-test-slave-9gcw"
server_key: "b70b1c78c9029383598c4949d7b77505"
server_key_iv: "54c4c3b878e745cb98f510fcbcf3b05e"
server_key_version: "encryptionkey@0"
I20260808 02:12:46.451772 20149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:46.507552 20149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:46.511308 20149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:46.511531 20149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:46.512238 20149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:46.513135 20149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:46.513265 20149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:46.513341 20149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:46.513387 20149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:46.523232 20149 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:46307
I20260808 02:12:46.523291 20287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:46307 every 8 connection(s)
I20260808 02:12:46.524233 20149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:12:46.528151 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 20149
I20260808 02:12:46.528314 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:12:46.528600 19170 external_mini_cluster.cc:1428] Setting key 9d213652e328b9a973a66363fd9d5f2f
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[19740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155166, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:46.539610 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.526594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:45099 (local address 127.18.184.190:41343)
0808 02:12:46.526866 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:46.526871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:46.527972 (+ 1101us) server_negotiation.cc:407] Connection header received
0808 02:12:46.529001 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:46.529004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:46.529081 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:46.529276 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:46.530544 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.531094 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.532316 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.532543 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.532653 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.535659 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:46.535663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:46.535665 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:46.535692 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:46.537361 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.537955 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.537958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.537961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.538051 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.538413 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.538417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.538419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.538641 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:46.538763 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:46.539300 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 02:12:46.539440 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260808 02:12:46.540570 20292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.526984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:41343 (local address 127.18.184.131:45099)
0808 02:12:46.527792 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:46.527840 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:46.528782 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:46.529459 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:46.529466 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:46.529882 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:46.530394 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.530409 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.531230 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.531239 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.532176 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.532186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.532712 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.532716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.532893 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.533549 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:46.533566 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:46.535450 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:46.537495 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:46.537502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:46.537524 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:46.537842 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:46.538153 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:46.538156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:46.538158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:46.538316 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:46.538762 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:46.538770 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:46.539043 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 02:12:46.539259 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 02:12:46.539641 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":135,"threads_started":1}
I20260808 02:12:46.542080 20289 heartbeater.cc:347] Connected to a master server at 127.18.184.190:41343
I20260808 02:12:46.542402 20289 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:46.543020 20289 heartbeater.cc:511] Master 127.18.184.190:41343 requested a full tablet report, sending...
I20260808 02:12:46.544200 19792 ts_manager.cc:194] Registered new tserver with Master: 2039ffe7dfbb4043bf2e8f88a7e2e441 (127.18.184.131:46307)
I20260808 02:12:46.544848 19792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:45099
I20260808 02:12:46.555164 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:12:46.567283 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.558957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59072 (local address 127.18.184.190:41343)
0808 02:12:46.559091 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:46.559094 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:46.559280 (+ 186us) server_negotiation.cc:407] Connection header received
0808 02:12:46.559328 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:46.559330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:46.559377 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:46.559439 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:46.560273 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.560869 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.561859 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.562119 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.562226 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.563737 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:46.563740 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:46.563742 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:46.563769 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:46.565587 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.566084 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.566086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.566088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.566169 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.566410 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.566412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.566414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.566705 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:46.566820 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:46.567051 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 02:12:46.567156 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:12:46.572630 19792 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:12:46.575932 19792 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:46.591415 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.585212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60832 (local address 127.18.184.131:46307)
0808 02:12:46.585619 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:46.585631 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:46.585655 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:12:46.585724 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:46.585729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:46.585926 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:46.586118 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:46.586768 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.587980 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.589723 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.590603 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.590744 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.590918 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:46.591124 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 02:12:46.591250 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":174,"threads_started":1}
I20260808 02:12:46.591411 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.585023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:46307 (local address 127.0.0.1:60832)
0808 02:12:46.585381 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:46.585429 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:46.585552 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:46.586374 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:46.586381 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:46.586410 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:46.586597 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.586608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.588124 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.588127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.589581 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.589590 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.590754 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.590759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.590884 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.590937 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 02:12:46.591075 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 02:12:46.591236 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":91,"threads_started":1}
I20260808 02:12:46.591895 20304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.585833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:38019 (local address 127.0.0.1:50680)
0808 02:12:46.586230 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:46.586246 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:46.586341 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:46.587209 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:46.587212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:46.587233 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:46.587367 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.587374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.589013 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.589020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.590184 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.590194 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.591413 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.591417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.591541 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.591575 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:12:46.591672 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 02:12:46.591752 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260808 02:12:46.592382 20305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.586032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50680 (local address 127.18.184.130:38019)
0808 02:12:46.586708 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:46.586725 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:46.586749 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:12:46.586832 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:46.586838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:46.587041 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:46.587206 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:46.588053 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.588862 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.590323 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.591086 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.591217 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.591342 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:46.592111 (+ 769us) server_negotiation.cc:299] Negotiation successful
0808 02:12:46.592197 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":142,"threads_started":1}
I20260808 02:12:46.592823 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.586626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41166 (local address 127.18.184.129:46825)
0808 02:12:46.587040 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:46.587048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:46.587315 (+ 267us) server_negotiation.cc:407] Connection header received
0808 02:12:46.587542 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:46.587548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:46.587742 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:46.587887 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:46.588095 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.589330 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.590870 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.591700 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.591810 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.591938 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:46.592290 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 02:12:46.592393 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":127,"threads_started":1}
I20260808 02:12:46.592413 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.586194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:46825 (local address 127.0.0.1:41166)
0808 02:12:46.587221 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:46.587234 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:46.587379 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:46.587877 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:46.587879 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:46.587895 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:46.587980 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.587984 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.589475 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.589479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.590726 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.590736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.592039 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.592043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.592136 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.592156 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:12:46.592229 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 02:12:46.592293 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"thread_start_us":107,"threads_started":1}
I20260808 02:12:46.594990 20221 tablet_service.cc:1467] Processing CreateTablet for tablet 13cee1c1b0f040b3a25f8f564b89851e (DEFAULT_TABLE table=test-table [id=2ca82556daba41ef835a57810b883876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:46.595537 20073 tablet_service.cc:1467] Processing CreateTablet for tablet 13cee1c1b0f040b3a25f8f564b89851e (DEFAULT_TABLE table=test-table [id=2ca82556daba41ef835a57810b883876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:46.595677 19929 tablet_service.cc:1467] Processing CreateTablet for tablet 13cee1c1b0f040b3a25f8f564b89851e (DEFAULT_TABLE table=test-table [id=2ca82556daba41ef835a57810b883876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:46.596284 20221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13cee1c1b0f040b3a25f8f564b89851e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:46.596601 20073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13cee1c1b0f040b3a25f8f564b89851e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:46.596822 19929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13cee1c1b0f040b3a25f8f564b89851e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:46.603286 20308 tablet_bootstrap.cc:492] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4: Bootstrap starting.
I20260808 02:12:46.604022 20309 tablet_bootstrap.cc:492] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd: Bootstrap starting.
I20260808 02:12:46.604490 20310 tablet_bootstrap.cc:492] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441: Bootstrap starting.
I20260808 02:12:46.606330 20310 tablet_bootstrap.cc:654] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:46.606604 20308 tablet_bootstrap.cc:654] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:46.606981 20309 tablet_bootstrap.cc:654] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:46.607618 20310 log.cc:824] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:46.607630 20308 log.cc:824] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:46.607818 20309 log.cc:824] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:46.609889 20310 tablet_bootstrap.cc:492] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441: No bootstrap required, opened a new log
I20260808 02:12:46.609898 20308 tablet_bootstrap.cc:492] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4: No bootstrap required, opened a new log
I20260808 02:12:46.609895 20309 tablet_bootstrap.cc:492] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd: No bootstrap required, opened a new log
I20260808 02:12:46.610098 20310 ts_tablet_manager.cc:1397] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260808 02:12:46.610102 20309 ts_tablet_manager.cc:1397] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:12:46.610114 20308 ts_tablet_manager.cc:1397] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:12:46.613039 20310 raft_consensus.cc:348] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 46307 } } peers { permanent_uuid: "320511051fc6496497eb7d430d7b5cbd" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38019 } } peers { permanent_uuid: "67585de9427e478c9f4995217ba42bf4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 46825 } }
I20260808 02:12:46.613366 20310 raft_consensus.cc:374] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:46.613454 20310 raft_consensus.cc:729] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2039ffe7dfbb4043bf2e8f88a7e2e441, State: Initialized, Role: FOLLOWER
I20260808 02:12:46.613947 20310 consensus_queue.cc:260] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 46307 } } peers { permanent_uuid: "320511051fc6496497eb7d430d7b5cbd" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38019 } } peers { permanent_uuid: "67585de9427e478c9f4995217ba42bf4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 46825 } }
I20260808 02:12:46.614411 20309 raft_consensus.cc:348] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 46307 } } peers { permanent_uuid: "320511051fc6496497eb7d430d7b5cbd" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38019 } } peers { permanent_uuid: "67585de9427e478c9f4995217ba42bf4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 46825 } }
I20260808 02:12:46.614737 20309 raft_consensus.cc:374] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:46.614826 20309 raft_consensus.cc:729] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320511051fc6496497eb7d430d7b5cbd, State: Initialized, Role: FOLLOWER
I20260808 02:12:46.614850 20308 raft_consensus.cc:348] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 46307 } } peers { permanent_uuid: "320511051fc6496497eb7d430d7b5cbd" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38019 } } peers { permanent_uuid: "67585de9427e478c9f4995217ba42bf4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 46825 } }
I20260808 02:12:46.615110 20310 ts_tablet_manager.cc:1428] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260808 02:12:46.615157 20308 raft_consensus.cc:374] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:46.615260 20308 raft_consensus.cc:729] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67585de9427e478c9f4995217ba42bf4, State: Initialized, Role: FOLLOWER
I20260808 02:12:46.615417 20289 heartbeater.cc:503] Master 127.18.184.190:41343 was elected leader, sending a full tablet report...
I20260808 02:12:46.615430 20309 consensus_queue.cc:260] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 46307 } } peers { permanent_uuid: "320511051fc6496497eb7d430d7b5cbd" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38019 } } peers { permanent_uuid: "67585de9427e478c9f4995217ba42bf4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 46825 } }
I20260808 02:12:46.615761 20308 consensus_queue.cc:260] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 46307 } } peers { permanent_uuid: "320511051fc6496497eb7d430d7b5cbd" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38019 } } peers { permanent_uuid: "67585de9427e478c9f4995217ba42bf4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 46825 } }
I20260808 02:12:46.616168 20143 heartbeater.cc:503] Master 127.18.184.190:41343 was elected leader, sending a full tablet report...
I20260808 02:12:46.616442 20309 ts_tablet_manager.cc:1428] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260808 02:12:46.617031 20308 ts_tablet_manager.cc:1428] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 02:12:46.617431 19998 heartbeater.cc:503] Master 127.18.184.190:41343 was elected leader, sending a full tablet report...
I20260808 02:12:46.645650 20316 raft_consensus.cc:482] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:46.645918 20316 raft_consensus.cc:504] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 46307 } } peers { permanent_uuid: "320511051fc6496497eb7d430d7b5cbd" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38019 } } peers { permanent_uuid: "67585de9427e478c9f4995217ba42bf4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 46825 } }
I20260808 02:12:46.647363 20316 leader_election.cc:288] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2039ffe7dfbb4043bf2e8f88a7e2e441 (127.18.184.131:46307), 67585de9427e478c9f4995217ba42bf4 (127.18.184.129:46825)
I20260808 02:12:46.651743 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.647599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:46307 (local address 127.18.184.130:57951)
0808 02:12:46.647755 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:46.647771 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:46.647877 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:46.648224 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:46.648227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:46.648249 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:46.648384 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.648392 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.649441 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.649445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.650449 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.650458 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.651288 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.651291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.651401 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.651426 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:12:46.651506 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 02:12:46.651583 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":21}
I20260808 02:12:46.651808 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.647758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:57951 (local address 127.18.184.131:46307)
0808 02:12:46.647931 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:46.647937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:46.647953 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:12:46.648012 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:46.648015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:46.648097 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:46.648211 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:46.648525 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.649331 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.650584 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.651172 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.651264 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.651329 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:46.651528 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 02:12:46.651587 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:12:46.652259 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.648187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:42461 (local address 127.18.184.129:46825)
0808 02:12:46.648351 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:46.648356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:46.648612 (+ 256us) server_negotiation.cc:407] Connection header received
0808 02:12:46.648702 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:46.648704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:46.648744 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:46.648815 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:46.649096 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.649803 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.651035 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.651689 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.651831 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.651884 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:46.652070 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 02:12:46.652148 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 02:12:46.652273 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.647994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:46825 (local address 127.18.184.130:42461)
0808 02:12:46.648509 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:46.648523 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:46.648621 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:46.648846 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:46.648849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:46.648865 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:46.648989 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.648995 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.649925 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.649928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.650903 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:46.650913 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.651829 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.651836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:46.651935 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.651966 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:12:46.652048 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 02:12:46.652112 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":1,"thread_start_us":141,"threads_started":1}
I20260808 02:12:46.652731 20241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13cee1c1b0f040b3a25f8f564b89851e" candidate_uuid: "320511051fc6496497eb7d430d7b5cbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441" is_pre_election: true
I20260808 02:12:46.653065 19949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13cee1c1b0f040b3a25f8f564b89851e" candidate_uuid: "320511051fc6496497eb7d430d7b5cbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67585de9427e478c9f4995217ba42bf4" is_pre_election: true
I20260808 02:12:46.653206 20241 raft_consensus.cc:2454] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 320511051fc6496497eb7d430d7b5cbd in term 0.
I20260808 02:12:46.653383 19949 raft_consensus.cc:2454] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 320511051fc6496497eb7d430d7b5cbd in term 0.
I20260808 02:12:46.653774 20024 leader_election.cc:302] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2039ffe7dfbb4043bf2e8f88a7e2e441, 320511051fc6496497eb7d430d7b5cbd; no voters:
I20260808 02:12:46.654047 20316 raft_consensus.cc:2781] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:46.654143 20316 raft_consensus.cc:482] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:46.654188 20316 raft_consensus.cc:3037] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:46.655319 20316 raft_consensus.cc:504] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 46307 } } peers { permanent_uuid: "320511051fc6496497eb7d430d7b5cbd" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38019 } } peers { permanent_uuid: "67585de9427e478c9f4995217ba42bf4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 46825 } }
I20260808 02:12:46.655669 20316 leader_election.cc:288] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [CANDIDATE]: Term 1 election: Requested vote from peers 2039ffe7dfbb4043bf2e8f88a7e2e441 (127.18.184.131:46307), 67585de9427e478c9f4995217ba42bf4 (127.18.184.129:46825)
I20260808 02:12:46.656114 19949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13cee1c1b0f040b3a25f8f564b89851e" candidate_uuid: "320511051fc6496497eb7d430d7b5cbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67585de9427e478c9f4995217ba42bf4"
I20260808 02:12:46.656143 20241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13cee1c1b0f040b3a25f8f564b89851e" candidate_uuid: "320511051fc6496497eb7d430d7b5cbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441"
I20260808 02:12:46.656225 19949 raft_consensus.cc:3037] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:46.656329 20241 raft_consensus.cc:3037] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:46.657608 19949 raft_consensus.cc:2454] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 320511051fc6496497eb7d430d7b5cbd in term 1.
I20260808 02:12:46.657874 20241 raft_consensus.cc:2454] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 320511051fc6496497eb7d430d7b5cbd in term 1.
I20260808 02:12:46.657936 20024 leader_election.cc:302] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320511051fc6496497eb7d430d7b5cbd, 67585de9427e478c9f4995217ba42bf4; no voters:
I20260808 02:12:46.658143 20316 raft_consensus.cc:2781] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:46.658385 20316 raft_consensus.cc:686] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [term 1 LEADER]: Becoming Leader. State: Replica: 320511051fc6496497eb7d430d7b5cbd, State: Running, Role: LEADER
I20260808 02:12:46.658747 20316 consensus_queue.cc:237] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 46307 } } peers { permanent_uuid: "320511051fc6496497eb7d430d7b5cbd" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38019 } } peers { permanent_uuid: "67585de9427e478c9f4995217ba42bf4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 46825 } }
I20260808 02:12:46.662214 19791 catalog_manager.cc:5495] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 320511051fc6496497eb7d430d7b5cbd (127.18.184.130). New cstate: current_term: 1 leader_uuid: "320511051fc6496497eb7d430d7b5cbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2039ffe7dfbb4043bf2e8f88a7e2e441" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 46307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "320511051fc6496497eb7d430d7b5cbd" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67585de9427e478c9f4995217ba42bf4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 46825 } health_report { overall_health: UNKNOWN } } }
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[19740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.129@KRBTEST.COM
I20260808 02:12:46.693913 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.683925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.18.184.129:46825)
0808 02:12:46.684075 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:46.684080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:46.684418 (+ 338us) server_negotiation.cc:407] Connection header received
0808 02:12:46.684569 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:46.684572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:46.684629 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:46.684717 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:46.685491 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.686007 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.686876 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.687098 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.687208 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.690055 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:46.690068 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:46.690083 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:46.690110 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:46.691861 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.692357 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.692363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.692364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.692442 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.692921 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.692925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.692926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.693277 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:46.693475 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:46.693671 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 02:12:46.693787 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[19740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155166, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[19740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155166, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.184.129@KRBTEST.COM
I20260808 02:12:46.723711 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.714540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.18.184.129:46825)
0808 02:12:46.714792 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:46.714798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:46.714907 (+ 109us) server_negotiation.cc:407] Connection header received
0808 02:12:46.715090 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:46.715094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:46.715138 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:46.715241 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:46.715982 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.716516 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.717448 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.717688 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.717797 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.720344 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:46.720348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:46.720350 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:46.720380 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:46.721875 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.722385 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.722388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.722390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.722486 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.722809 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.722812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.722813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.723001 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:46.723113 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:46.723440 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 02:12:46.723555 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[19740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260808 02:12:46.749318 20144 tablet.cc:2369] T 13cee1c1b0f040b3a25f8f564b89851e P 320511051fc6496497eb7d430d7b5cbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[19740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.129@KRBTEST.COM
I20260808 02:12:46.755025 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:46.744295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41190 (local address 127.18.184.129:46825)
0808 02:12:46.744473 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:46.744478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:46.744654 (+ 176us) server_negotiation.cc:407] Connection header received
0808 02:12:46.744918 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:46.744922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:46.744983 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:46.745087 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:46.746040 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.746636 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.747608 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:46.747863 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:46.747988 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:46.751304 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:46.751314 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:46.751316 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:46.751344 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:46.752995 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.753567 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.753570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.753572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.753662 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:46.754049 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:46.754053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:46.754055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:46.754293 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:46.754413 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:46.754777 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 02:12:46.754894 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 02:12:46.756594 19905 tablet_service.cc:2908] Rejecting scan request for tablet 13cee1c1b0f040b3a25f8f564b89851e: Uninitialized: clean time has not yet been initialized
W20260808 02:12:46.757552 19999 tablet.cc:2369] T 13cee1c1b0f040b3a25f8f564b89851e P 67585de9427e478c9f4995217ba42bf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:12:46.758929 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 19860
I20260808 02:12:46.770124 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 20005
W20260808 02:12:46.775378 20291 tablet.cc:2369] T 13cee1c1b0f040b3a25f8f564b89851e P 2039ffe7dfbb4043bf2e8f88a7e2e441: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:12:46.778021 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 20149
I20260808 02:12:46.786859 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 19758
2026-08-08T02:12:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2576 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[20335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[20335](info): set up 2 sockets
Aug 08 02:12:46 dist-test-slave-9gcw krb5kdc[20335](info): commencing operation
krb5kdc: starting...
W20260808 02:12:47.886456 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:12:47 dist-test-slave-9gcw krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:12:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:12:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:12:48.023823 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:37395
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:37395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:48.139918 20348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:48.140295 20348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:48.140388 20348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:48.144577 20348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:12:48.144698 20348 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:12:48.144727 20348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:48.144759 20348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:48.144800 20348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:12:48.144863 20348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:12:48.150514 20348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:37395
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:37395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:48.151754 20348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:48.152864 20348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:48.162240 20357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:48.162261 20354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:48.162305 20353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:48.162822 20348 server_base.cc:1034] running on GCE node
I20260808 02:12:48.163406 20348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:48.164705 20348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:48.165885 20348 hybrid_clock.cc:648] HybridClock initialized: now 1786155168165863 us; error 32 us; skew 500 ppm
Aug 08 02:12:48 dist-test-slave-9gcw krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155168, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:48.173409 20348 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:12:48.174458 20348 webserver.cc:460] Webserver started at http://127.18.184.190:44437/ using document root <none> and password file <none>
I20260808 02:12:48.175046 20348 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:48.175150 20348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:48.175415 20348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:48.177625 20348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "5ab5a034dbee4188927452d1a8b22286"
format_stamp: "Formatted at 2026-08-08 02:12:48 on dist-test-slave-9gcw"
server_key: "b86896f7e71732ddebdd2470ef452a64"
server_key_iv: "fde69889537f91a14f948fef65a7c96f"
server_key_version: "encryptionkey@0"
I20260808 02:12:48.178206 20348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "5ab5a034dbee4188927452d1a8b22286"
format_stamp: "Formatted at 2026-08-08 02:12:48 on dist-test-slave-9gcw"
server_key: "b86896f7e71732ddebdd2470ef452a64"
server_key_iv: "fde69889537f91a14f948fef65a7c96f"
server_key_version: "encryptionkey@0"
I20260808 02:12:48.182113 20348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:12:48.184640 20365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:48.186022 20348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:12:48.186172 20348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "5ab5a034dbee4188927452d1a8b22286"
format_stamp: "Formatted at 2026-08-08 02:12:48 on dist-test-slave-9gcw"
server_key: "b86896f7e71732ddebdd2470ef452a64"
server_key_iv: "fde69889537f91a14f948fef65a7c96f"
server_key_version: "encryptionkey@0"
I20260808 02:12:48.186286 20348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:48.211942 20348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:48.215682 20348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:48.215942 20348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:48.225653 20348 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:37395
I20260808 02:12:48.225739 20431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:37395 every 8 connection(s)
I20260808 02:12:48.226752 20348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:12:48.229861 20432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:48.231959 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 20348
I20260808 02:12:48.232091 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:12:48.232360 19170 external_mini_cluster.cc:1428] Setting key 9242bcddcd3d18f7c1f70e5ac56f004e
I20260808 02:12:48.236640 20432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286: Bootstrap starting.
I20260808 02:12:48.239476 20432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:48.240393 20432 log.cc:824] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:12:48 dist-test-slave-9gcw krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:48.242950 20432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286: No bootstrap required, opened a new log
I20260808 02:12:48.247864 20432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab5a034dbee4188927452d1a8b22286" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37395 } }
I20260808 02:12:48.248183 20432 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:48.248250 20432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab5a034dbee4188927452d1a8b22286, State: Initialized, Role: FOLLOWER
I20260808 02:12:48.248775 20432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab5a034dbee4188927452d1a8b22286" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37395 } }
I20260808 02:12:48.248947 20432 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:12:48.249008 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:48.234030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57508 (local address 127.18.184.190:37395)
0808 02:12:48.234483 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:48.234506 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:48.234553 (+ 47us) server_negotiation.cc:407] Connection header received
0808 02:12:48.235465 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:48.235489 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:48.235909 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:48.236286 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:48.237040 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.238552 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.239449 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.240152 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.240340 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:48.243495 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:48.243504 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:48.243528 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:48.243572 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:48.245668 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:48.246128 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:48.246135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:48.246141 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:48.246294 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:48.246625 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:48.246628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:48.246630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:48.247200 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:48.247429 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:48.247641 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 02:12:48.247948 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":118,"threads_started":1}
I20260808 02:12:48.249017 20432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:12:48.249186 20432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:48.250214 20432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab5a034dbee4188927452d1a8b22286" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37395 } }
I20260808 02:12:48.250530 20432 leader_election.cc:302] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ab5a034dbee4188927452d1a8b22286; no voters:
I20260808 02:12:48.250864 20432 leader_election.cc:288] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:12:48.251014 20438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:48.251237 20438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [term 1 LEADER]: Becoming Leader. State: Replica: 5ab5a034dbee4188927452d1a8b22286, State: Running, Role: LEADER
I20260808 02:12:48.251564 20438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab5a034dbee4188927452d1a8b22286" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37395 } }
I20260808 02:12:48.252277 20432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:12:48.253485 20439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ab5a034dbee4188927452d1a8b22286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab5a034dbee4188927452d1a8b22286" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37395 } } }
I20260808 02:12:48.253512 20440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ab5a034dbee4188927452d1a8b22286. Latest consensus state: current_term: 1 leader_uuid: "5ab5a034dbee4188927452d1a8b22286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab5a034dbee4188927452d1a8b22286" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37395 } } }
I20260808 02:12:48.253648 20439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:48.253651 20440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:48.254174 20447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:12:48.257860 20447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:12:48.264784 20447 catalog_manager.cc:1347] Generated new cluster ID: 6382ffb63d9e4f9ab2c5ff9f13794f30
I20260808 02:12:48.264874 20447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:12:48.294090 20447 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:12:48.295042 20447 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:12:48.300635 20447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286: Generated new TSK 0
I20260808 02:12:48.301412 20447 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:12:48.341455 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:37395
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:48.458916 20460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:48.459244 20460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:48.459354 20460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:48.463522 20460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:48.463619 20460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:48.463698 20460 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:12:48.463783 20460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:12:48.469430 20460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.184.190:37395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:48.470598 20460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:48.471664 20460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:48.482043 20465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:48.482081 20468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:48.482087 20466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:48.482271 20460 server_base.cc:1034] running on GCE node
I20260808 02:12:48.482856 20460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:48.483520 20460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:48.484694 20460 hybrid_clock.cc:648] HybridClock initialized: now 1786155168484684 us; error 33 us; skew 500 ppm
Aug 08 02:12:48 dist-test-slave-9gcw krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155168, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:48.487833 20460 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:12:48.488785 20460 webserver.cc:460] Webserver started at http://127.18.184.129:40271/ using document root <none> and password file <none>
I20260808 02:12:48.489464 20460 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:48.489562 20460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:48.489836 20460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:48.492178 20460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "c5c6eb4464af44eaa920c42f3db6fb48"
format_stamp: "Formatted at 2026-08-08 02:12:48 on dist-test-slave-9gcw"
server_key: "305af646442025221fefe245e0a051d4"
server_key_iv: "97eb207498475423befdb3502eaf133d"
server_key_version: "encryptionkey@0"
I20260808 02:12:48.492781 20460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "c5c6eb4464af44eaa920c42f3db6fb48"
format_stamp: "Formatted at 2026-08-08 02:12:48 on dist-test-slave-9gcw"
server_key: "305af646442025221fefe245e0a051d4"
server_key_iv: "97eb207498475423befdb3502eaf133d"
server_key_version: "encryptionkey@0"
I20260808 02:12:48.496841 20460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260808 02:12:48.499416 20475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:48.500795 20460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:12:48.500953 20460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "c5c6eb4464af44eaa920c42f3db6fb48"
format_stamp: "Formatted at 2026-08-08 02:12:48 on dist-test-slave-9gcw"
server_key: "305af646442025221fefe245e0a051d4"
server_key_iv: "97eb207498475423befdb3502eaf133d"
server_key_version: "encryptionkey@0"
I20260808 02:12:48.501075 20460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:48.521080 20460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:48.524545 20460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:48.524791 20460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:48.526345 20460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:48.526422 20460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:48.526497 20460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:48.526542 20460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:12:48 dist-test-slave-9gcw krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155168, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:48.538725 20460 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:34377
I20260808 02:12:48.538862 20596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:34377 every 8 connection(s)
I20260808 02:12:48.539855 20460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:12:48.541337 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:48.527456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:46113 (local address 127.18.184.190:37395)
0808 02:12:48.527590 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:48.527594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:48.528716 (+ 1122us) server_negotiation.cc:407] Connection header received
0808 02:12:48.530023 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:48.530027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:48.530094 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:48.530163 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:48.531643 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.532255 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.533678 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.533990 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.534127 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:48.537060 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:48.537064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:48.537068 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:48.537100 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:48.538978 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:48.539464 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:48.539470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:48.539475 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:48.539571 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:48.539957 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:48.539961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:48.539962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:48.540229 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:48.540347 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:48.540988 (+ 641us) server_negotiation.cc:299] Negotiation successful
0808 02:12:48.541128 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:12:48.542088 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:48.528041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:37395 (local address 127.18.184.129:46113)
0808 02:12:48.528556 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:48.528600 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:48.529744 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:48.530496 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:48.530502 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:48.530833 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:48.531479 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:48.531503 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.532390 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.532393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:48.533536 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:48.533547 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.534189 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.534194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:48.534397 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:48.535065 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:48.535095 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:48.536885 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:48.539087 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:48.539091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:48.539102 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:48.539353 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:48.539712 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:48.539714 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:48.539716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:48.539850 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:48.540353 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:48.540360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:48.540719 (+ 359us) client_negotiation.cc:769] Sending connection context
0808 02:12:48.540938 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 02:12:48.541288 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":146,"threads_started":1}
I20260808 02:12:48.543438 20597 heartbeater.cc:347] Connected to a master server at 127.18.184.190:37395
I20260808 02:12:48.543620 20597 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:48.544301 20597 heartbeater.cc:511] Master 127.18.184.190:37395 requested a full tablet report, sending...
I20260808 02:12:48.546164 20381 ts_manager.cc:194] Registered new tserver with Master: c5c6eb4464af44eaa920c42f3db6fb48 (127.18.184.129:34377)
I20260808 02:12:48.547366 20381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:46113
I20260808 02:12:48.549901 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 20460
I20260808 02:12:48.550009 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:12:48.550319 19170 external_mini_cluster.cc:1428] Setting key 1a70dc6c6e0a0f0835c5c86fca8a7bfe
I20260808 02:12:48.555980 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:48.546171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:37395 (local address 127.18.184.129:37355)
0808 02:12:48.547713 (+ 1542us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:48.547728 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:48.547863 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:48.548135 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:48.548145 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:48.548387 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:48.548542 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:48.548549 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.549446 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.549449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:48.550272 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:48.550285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.550792 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.550795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:48.550894 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:48.551454 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:48.551473 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:48.551986 (+ 513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:48.554203 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:48.554206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:48.554208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:48.554407 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:48.554884 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:48.554886 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:48.554887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:48.554963 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:48.555520 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:48.555522 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:48.555606 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 02:12:48.555706 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 02:12:48.555833 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1461,"mutex_wait_us":2,"thread_start_us":168,"threads_started":1}
I20260808 02:12:48.556103 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:48.546337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:37355 (local address 127.18.184.190:37395)
0808 02:12:48.546459 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:48.546462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:48.547817 (+ 1355us) server_negotiation.cc:407] Connection header received
0808 02:12:48.547975 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:48.547979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:48.548030 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:48.548117 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:48.548737 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.549307 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.550419 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.550644 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.550778 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:48.552109 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:48.552112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:48.552114 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:48.552136 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:48.553979 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:48.554521 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:48.554527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:48.554530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:48.554760 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:48.555064 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:48.555071 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:48.555074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:48.555404 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:48.555547 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:48.555800 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 02:12:48.555916 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:12:48.584749 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:37395
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:48.712617 20606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:48.712946 20606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:48.713025 20606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:48.717244 20606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:48.717341 20606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:48.717417 20606 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:12:48.717504 20606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:12:48.723059 20606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.184.190:37395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:48.724293 20606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:48.725534 20606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:12:48.736218 20606 server_base.cc:1034] running on GCE node
W20260808 02:12:48.736148 20612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:48.736081 20614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:48.736075 20611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:48.736938 20606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:48.737697 20606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:48.738874 20606 hybrid_clock.cc:648] HybridClock initialized: now 1786155168738843 us; error 49 us; skew 500 ppm
Aug 08 02:12:48 dist-test-slave-9gcw krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155168, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:48.742105 20606 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:12:48.743148 20606 webserver.cc:460] Webserver started at http://127.18.184.130:40903/ using document root <none> and password file <none>
I20260808 02:12:48.744053 20606 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:48.744148 20606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:48.744401 20606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:48.747110 20606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "5d230d00b0cc496fba0bbfd0b423855c"
format_stamp: "Formatted at 2026-08-08 02:12:48 on dist-test-slave-9gcw"
server_key: "b5940c666ab74ebd796d190285071ee1"
server_key_iv: "a171360e31bf4aa077c82032ade521b3"
server_key_version: "encryptionkey@0"
I20260808 02:12:48.747758 20606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "5d230d00b0cc496fba0bbfd0b423855c"
format_stamp: "Formatted at 2026-08-08 02:12:48 on dist-test-slave-9gcw"
server_key: "b5940c666ab74ebd796d190285071ee1"
server_key_iv: "a171360e31bf4aa077c82032ade521b3"
server_key_version: "encryptionkey@0"
I20260808 02:12:48.752730 20606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:12:48.756031 20622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:48.757588 20606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 02:12:48.757748 20606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "5d230d00b0cc496fba0bbfd0b423855c"
format_stamp: "Formatted at 2026-08-08 02:12:48 on dist-test-slave-9gcw"
server_key: "b5940c666ab74ebd796d190285071ee1"
server_key_iv: "a171360e31bf4aa077c82032ade521b3"
server_key_version: "encryptionkey@0"
I20260808 02:12:48.757881 20606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:48.772562 20606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:48.776515 20606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:48.776751 20606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:48.778381 20606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:48.778468 20606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:48.778548 20606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:48.778600 20606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:12:48 dist-test-slave-9gcw krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155168, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:48.790315 20606 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:41927
I20260808 02:12:48.791397 20606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:12:48.792268 20748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:41927 every 8 connection(s)
I20260808 02:12:48.793253 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 20606
I20260808 02:12:48.793375 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:12:48.793650 19170 external_mini_cluster.cc:1428] Setting key 9fbe264c409d649753473328af2d34cb
I20260808 02:12:48.794848 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:48.779474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:45183 (local address 127.18.184.190:37395)
0808 02:12:48.779630 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:48.779634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:48.780838 (+ 1204us) server_negotiation.cc:407] Connection header received
0808 02:12:48.782150 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:48.782157 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:48.782233 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:48.782336 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:48.783838 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.784760 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.786231 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.786644 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.786774 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:48.790271 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:48.790276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:48.790279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:48.790316 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:48.792209 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:48.792791 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:48.792794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:48.792796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:48.792890 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:48.793318 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:48.793321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:48.793323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:48.793620 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:48.793752 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:48.794541 (+ 789us) server_negotiation.cc:299] Negotiation successful
0808 02:12:48.794659 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:12:48.795713 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:48.780090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:37395 (local address 127.18.184.130:45183)
0808 02:12:48.780645 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:48.780697 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:48.781877 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:48.782563 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:48.782573 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:48.783064 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:48.783684 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:48.783702 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.784927 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.784930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:48.786095 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:48.786106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.786848 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.786853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:48.787104 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:48.787826 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:48.787843 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:48.790038 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:48.792330 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:48.792335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:48.792351 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:48.792670 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:48.793001 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:48.793004 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:48.793006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:48.793205 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:48.793748 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:48.793754 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:48.794041 (+ 287us) client_negotiation.cc:769] Sending connection context
0808 02:12:48.794249 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 02:12:48.794631 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":151,"threads_started":1}
I20260808 02:12:48.797338 20750 heartbeater.cc:347] Connected to a master server at 127.18.184.190:37395
I20260808 02:12:48.797487 20750 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:48.798085 20750 heartbeater.cc:511] Master 127.18.184.190:37395 requested a full tablet report, sending...
I20260808 02:12:48.799270 20382 ts_manager.cc:194] Registered new tserver with Master: 5d230d00b0cc496fba0bbfd0b423855c (127.18.184.130:41927)
I20260808 02:12:48.800081 20382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:45183
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:12:48.808916 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:48.799927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:37395 (local address 127.18.184.130:45613)
0808 02:12:48.800501 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:48.800518 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:48.800679 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:48.801095 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:48.801098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:48.801364 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:48.801485 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:48.801494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.802554 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.802556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:48.803195 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:48.803209 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.803665 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.803671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:48.803801 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:48.804419 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:48.804439 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:48.804956 (+ 517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:48.806972 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:48.806975 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:48.806978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:48.807259 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:48.807657 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:48.807659 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:48.807660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:48.807742 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:48.808352 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:48.808354 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:48.808443 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 02:12:48.808531 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 02:12:48.808759 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":178,"threads_started":1}
I20260808 02:12:48.808960 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:48.800031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:45613 (local address 127.18.184.190:37395)
0808 02:12:48.800190 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:48.800197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:48.800589 (+ 392us) server_negotiation.cc:407] Connection header received
0808 02:12:48.800933 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:48.800936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:48.800979 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:48.801048 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:48.801647 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.802389 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.803310 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:48.803526 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:48.803610 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:48.805111 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:48.805115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:48.805117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:48.805140 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:48.806851 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:48.807456 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:48.807460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:48.807461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:48.807544 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:48.807876 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:48.807880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:48.807881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:48.808179 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:48.808367 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:48.808701 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 02:12:48.808828 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:12:48.829356 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:37395
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:48.953729 20759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:48.954041 20759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:48.954130 20759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:48.958359 20759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:48.958458 20759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:48.958534 20759 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:12:48.958616 20759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:12:48.964242 20759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.184.190:37395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:48.965476 20759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:48.966547 20759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:48.977486 20768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:48.977488 20765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:48.977488 20764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:48.977874 20759 server_base.cc:1034] running on GCE node
I20260808 02:12:48.978468 20759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:48.979166 20759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:48.980403 20759 hybrid_clock.cc:648] HybridClock initialized: now 1786155168980382 us; error 35 us; skew 500 ppm
Aug 08 02:12:48 dist-test-slave-9gcw krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155168, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:48.983579 20759 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:12:48.984700 20759 webserver.cc:460] Webserver started at http://127.18.184.131:44219/ using document root <none> and password file <none>
I20260808 02:12:48.985399 20759 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:48.985503 20759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:48.985774 20759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:48.988081 20759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "686779813126482b87ffb739fe669426"
format_stamp: "Formatted at 2026-08-08 02:12:48 on dist-test-slave-9gcw"
server_key: "08a3dce706639ce101c5673c20849727"
server_key_iv: "87d2a7b3b206c412ec0e279aa61c2a1c"
server_key_version: "encryptionkey@0"
I20260808 02:12:48.988747 20759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "686779813126482b87ffb739fe669426"
format_stamp: "Formatted at 2026-08-08 02:12:48 on dist-test-slave-9gcw"
server_key: "08a3dce706639ce101c5673c20849727"
server_key_iv: "87d2a7b3b206c412ec0e279aa61c2a1c"
server_key_version: "encryptionkey@0"
I20260808 02:12:48.993321 20759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260808 02:12:48.996150 20775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:48.997920 20759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:12:48.998095 20759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "686779813126482b87ffb739fe669426"
format_stamp: "Formatted at 2026-08-08 02:12:48 on dist-test-slave-9gcw"
server_key: "08a3dce706639ce101c5673c20849727"
server_key_iv: "87d2a7b3b206c412ec0e279aa61c2a1c"
server_key_version: "encryptionkey@0"
I20260808 02:12:48.998235 20759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:49.020167 20759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:49.024097 20759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:49.024375 20759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:49.026223 20759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:49.026305 20759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:49.026388 20759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:49.026446 20759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:12:49 dist-test-slave-9gcw krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155168, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:49.038995 20759 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:33885
I20260808 02:12:49.039052 20897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:33885 every 8 connection(s)
I20260808 02:12:49.040024 20759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:12:49.041834 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.027163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:49391 (local address 127.18.184.190:37395)
0808 02:12:49.027348 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.027353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.028458 (+ 1105us) server_negotiation.cc:407] Connection header received
0808 02:12:49.029964 (+ 1506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.029968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.030035 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.030165 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:49.031651 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.032203 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.033503 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.033744 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.033868 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.037364 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:49.037369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:49.037371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:49.037405 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:49.039312 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:49.039851 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:49.039855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:49.039857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:49.039967 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:49.040390 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:49.040393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:49.040395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:49.040653 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:49.040760 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.041465 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.041696 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:12:49.042512 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.027682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:37395 (local address 127.18.184.131:49391)
0808 02:12:49.028241 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:49.028313 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:49.029664 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:49.030396 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:49.030406 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:49.030825 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:49.031467 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.031491 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.032349 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.032353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.033370 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.033379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.033914 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.033918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.034128 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.034794 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:49.034811 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:49.037063 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:49.039439 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:49.039445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:49.039460 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:49.039733 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:49.040064 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:49.040068 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:49.040070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:49.040287 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:49.040816 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:49.040823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:49.041122 (+ 299us) client_negotiation.cc:769] Sending connection context
0808 02:12:49.041331 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 02:12:49.041596 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":118,"threads_started":1}
I20260808 02:12:49.043977 20898 heartbeater.cc:347] Connected to a master server at 127.18.184.190:37395
I20260808 02:12:49.044185 20898 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:49.044909 20898 heartbeater.cc:511] Master 127.18.184.190:37395 requested a full tablet report, sending...
I20260808 02:12:49.046131 20381 ts_manager.cc:194] Registered new tserver with Master: 686779813126482b87ffb739fe669426 (127.18.184.131:33885)
I20260808 02:12:49.046845 20381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:49391
I20260808 02:12:49.047573 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 20759
I20260808 02:12:49.047713 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:12:49.048022 19170 external_mini_cluster.cc:1428] Setting key 2289f6cd2c49b6cb2bef4d160aaebd0d
I20260808 02:12:49.050312 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:12:49.054575 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.046354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:46471 (local address 127.18.184.190:37395)
0808 02:12:49.046489 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.046493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.046802 (+ 309us) server_negotiation.cc:407] Connection header received
0808 02:12:49.046920 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.046926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.046972 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.047058 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:49.047653 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.048378 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.049495 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.049731 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.049843 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.051097 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:49.051101 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:49.051103 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:49.051128 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:49.052688 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:49.053129 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:49.053133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:49.053135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:49.053326 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:49.053688 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:49.053692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:49.053693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:49.053890 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:49.053999 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.054280 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.054444 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:12:49.054586 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.046328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:37395 (local address 127.18.184.131:46471)
0808 02:12:49.046709 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:49.046724 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:49.046827 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:49.047084 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:49.047089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:49.047352 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:49.047507 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.047518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.048514 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.048517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.049368 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.049379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.049848 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.049852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.049945 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.050567 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:49.050579 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:49.050962 (+ 383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:49.052814 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:49.052818 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:49.052820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:49.053025 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:49.053466 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:49.053469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:49.053471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:49.053562 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:49.053987 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:49.053990 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:49.054096 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 02:12:49.054204 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 02:12:49.054380 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":103,"threads_started":1}
Aug 08 02:12:49 dist-test-slave-9gcw krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:12:49 dist-test-slave-9gcw krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:49.082487 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.072360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57518 (local address 127.18.184.190:37395)
0808 02:12:49.072514 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.072518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.072533 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:12:49.072578 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.072580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.072621 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.072697 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:49.073771 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.074324 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.075402 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.075611 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.075704 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.078647 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:49.078652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:49.078655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:49.078689 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:49.080406 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:49.080972 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:49.080975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:49.080977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:49.081069 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:49.081526 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:49.081528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:49.081530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:49.081760 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:49.081880 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.082117 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.082248 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:12:49.086025 20381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:12:49.089461 20381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:49.106264 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.100260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34444 (local address 127.18.184.130:41927)
0808 02:12:49.101056 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.101070 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.101092 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:12:49.101205 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.101213 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.101406 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.101893 (+ 487us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:49.102034 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.103206 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.104773 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.105451 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.105724 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.105908 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.105969 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.106082 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":132,"threads_started":1}
I20260808 02:12:49.106248 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.100060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:41927 (local address 127.0.0.1:34444)
0808 02:12:49.100334 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:49.100350 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:49.100451 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:49.101579 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:49.101582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:49.101601 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:49.101727 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.101734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.103329 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.103332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.104627 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.104642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.105600 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.105605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.105710 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.105745 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:12:49.105821 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:12:49.105985 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":88,"thread_start_us":108,"threads_started":1}
I20260808 02:12:49.106490 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.099273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:33885 (local address 127.0.0.1:38084)
0808 02:12:49.099807 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:49.099869 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:49.100020 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:49.100721 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:49.100733 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:49.100764 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:49.100938 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.100946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.102715 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.102718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.104253 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.104265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.105498 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.105502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.105633 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.105690 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 02:12:49.105791 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 02:12:49.105981 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
I20260808 02:12:49.106659 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.100410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:34377 (local address 127.0.0.1:51242)
0808 02:12:49.100692 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:49.100707 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:49.100782 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:49.101785 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:49.101788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:49.101810 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:49.101919 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.101930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.103143 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.103150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.104622 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.104634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.105981 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.105985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.106089 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.106127 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 02:12:49.106180 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 02:12:49.106273 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":93,"thread_start_us":112,"threads_started":1}
I20260808 02:12:49.106693 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.099483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38084 (local address 127.18.184.131:33885)
0808 02:12:49.099984 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.099998 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.100026 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:12:49.100281 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.100290 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.100525 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.100713 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:49.101092 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.102562 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.104407 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.105341 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.106226 (+ 885us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.106374 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.106440 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.106539 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":125,"threads_started":1}
I20260808 02:12:49.107481 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.100999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51242 (local address 127.18.184.129:34377)
0808 02:12:49.101299 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.101308 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.101332 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:12:49.101414 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.101419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.101615 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.101800 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:49.102063 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.102958 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.104806 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.105839 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.106963 (+ 1124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.107145 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.107205 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.107320 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":102,"threads_started":1}
I20260808 02:12:49.110354 20526 tablet_service.cc:1467] Processing CreateTablet for tablet eef8974ad3f542a1ade4a3685f4042c4 (DEFAULT_TABLE table=test-table [id=8e0b4d578457480ca0271d87cdf7b994]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:49.110442 20678 tablet_service.cc:1467] Processing CreateTablet for tablet eef8974ad3f542a1ade4a3685f4042c4 (DEFAULT_TABLE table=test-table [id=8e0b4d578457480ca0271d87cdf7b994]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:49.110689 20829 tablet_service.cc:1467] Processing CreateTablet for tablet eef8974ad3f542a1ade4a3685f4042c4 (DEFAULT_TABLE table=test-table [id=8e0b4d578457480ca0271d87cdf7b994]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:49.111369 20526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef8974ad3f542a1ade4a3685f4042c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:49.111598 20678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef8974ad3f542a1ade4a3685f4042c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:49.111603 20829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef8974ad3f542a1ade4a3685f4042c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:49.118383 20922 tablet_bootstrap.cc:492] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48: Bootstrap starting.
I20260808 02:12:49.120160 20924 tablet_bootstrap.cc:492] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426: Bootstrap starting.
I20260808 02:12:49.121183 20922 tablet_bootstrap.cc:654] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:49.121345 20923 tablet_bootstrap.cc:492] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c: Bootstrap starting.
I20260808 02:12:49.122205 20922 log.cc:824] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:49.122784 20924 tablet_bootstrap.cc:654] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:49.123546 20923 tablet_bootstrap.cc:654] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:49.123785 20924 log.cc:824] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:49.124497 20923 log.cc:824] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:49.125059 20922 tablet_bootstrap.cc:492] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48: No bootstrap required, opened a new log
I20260808 02:12:49.125340 20922 ts_tablet_manager.cc:1397] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:12:49.126045 20924 tablet_bootstrap.cc:492] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426: No bootstrap required, opened a new log
I20260808 02:12:49.126263 20924 ts_tablet_manager.cc:1397] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:12:49.126541 20923 tablet_bootstrap.cc:492] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c: No bootstrap required, opened a new log
I20260808 02:12:49.126756 20923 ts_tablet_manager.cc:1397] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:12:49.130113 20923 raft_consensus.cc:348] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } }
I20260808 02:12:49.130441 20923 raft_consensus.cc:374] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:49.130501 20923 raft_consensus.cc:729] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d230d00b0cc496fba0bbfd0b423855c, State: Initialized, Role: FOLLOWER
I20260808 02:12:49.130561 20924 raft_consensus.cc:348] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } }
I20260808 02:12:49.130561 20922 raft_consensus.cc:348] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } }
I20260808 02:12:49.130836 20922 raft_consensus.cc:374] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:49.130834 20924 raft_consensus.cc:374] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:49.130903 20924 raft_consensus.cc:729] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 686779813126482b87ffb739fe669426, State: Initialized, Role: FOLLOWER
I20260808 02:12:49.130903 20922 raft_consensus.cc:729] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c6eb4464af44eaa920c42f3db6fb48, State: Initialized, Role: FOLLOWER
I20260808 02:12:49.131059 20923 consensus_queue.cc:260] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } }
I20260808 02:12:49.131362 20924 consensus_queue.cc:260] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } }
I20260808 02:12:49.131363 20922 consensus_queue.cc:260] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } }
I20260808 02:12:49.132113 20750 heartbeater.cc:503] Master 127.18.184.190:37395 was elected leader, sending a full tablet report...
I20260808 02:12:49.132398 20924 ts_tablet_manager.cc:1428] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:12:49.132727 20922 ts_tablet_manager.cc:1428] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 02:12:49.132771 20597 heartbeater.cc:503] Master 127.18.184.190:37395 was elected leader, sending a full tablet report...
I20260808 02:12:49.132979 20898 heartbeater.cc:503] Master 127.18.184.190:37395 was elected leader, sending a full tablet report...
I20260808 02:12:49.133276 20923 ts_tablet_manager.cc:1428] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:12:49.172465 20930 raft_consensus.cc:482] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:49.172878 20930 raft_consensus.cc:504] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } }
I20260808 02:12:49.174263 20930 leader_election.cc:288] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 686779813126482b87ffb739fe669426 (127.18.184.131:33885), 5d230d00b0cc496fba0bbfd0b423855c (127.18.184.130:41927)
I20260808 02:12:49.179315 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.175196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:41927 (local address 127.18.184.129:42461)
0808 02:12:49.175545 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:49.175560 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:49.175675 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:49.175978 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:49.175980 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:49.175995 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:49.176099 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.176104 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.176941 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.176947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.177929 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.177939 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.178783 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.178787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.178887 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.178914 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:12:49.179002 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 02:12:49.179108 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":1,"thread_start_us":124,"threads_started":1}
I20260808 02:12:49.179333 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.175429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:42461 (local address 127.18.184.130:41927)
0808 02:12:49.175571 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.175578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.175626 (+ 48us) server_negotiation.cc:407] Connection header received
0808 02:12:49.175836 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.175840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.175887 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.175950 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:49.176246 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.176816 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.178069 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.178553 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.178697 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.178768 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.179076 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.179181 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:12:49.179381 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.174888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:48867 (local address 127.18.184.131:33885)
0808 02:12:49.175091 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.175098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.175120 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:12:49.175211 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.175214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.175277 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.175392 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:49.176061 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.176632 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.177949 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.178413 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.178617 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.178699 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.179138 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.179231 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:12:49.179780 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.174507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:33885 (local address 127.18.184.129:48867)
0808 02:12:49.174927 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:49.174947 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:49.175083 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:49.175509 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:49.175513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:49.175535 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:49.175671 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.175677 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.176758 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.176765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.177800 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.177811 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.178608 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.178612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.178731 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.179003 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 02:12:49.179540 (+ 537us) client_negotiation.cc:240] Negotiation successful
0808 02:12:49.179635 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":47,"thread_start_us":130,"threads_started":1}
I20260808 02:12:49.180342 20848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef8974ad3f542a1ade4a3685f4042c4" candidate_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "686779813126482b87ffb739fe669426" is_pre_election: true
I20260808 02:12:49.180335 20700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef8974ad3f542a1ade4a3685f4042c4" candidate_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d230d00b0cc496fba0bbfd0b423855c" is_pre_election: true
I20260808 02:12:49.180670 20700 raft_consensus.cc:2454] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5c6eb4464af44eaa920c42f3db6fb48 in term 0.
I20260808 02:12:49.180681 20848 raft_consensus.cc:2454] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5c6eb4464af44eaa920c42f3db6fb48 in term 0.
I20260808 02:12:49.181288 20480 leader_election.cc:302] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d230d00b0cc496fba0bbfd0b423855c, c5c6eb4464af44eaa920c42f3db6fb48; no voters:
I20260808 02:12:49.181598 20930 raft_consensus.cc:2781] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:49.181710 20930 raft_consensus.cc:482] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:49.181779 20930 raft_consensus.cc:3037] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:49.182950 20930 raft_consensus.cc:504] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } }
I20260808 02:12:49.183334 20930 leader_election.cc:288] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [CANDIDATE]: Term 1 election: Requested vote from peers 686779813126482b87ffb739fe669426 (127.18.184.131:33885), 5d230d00b0cc496fba0bbfd0b423855c (127.18.184.130:41927)
I20260808 02:12:49.183806 20700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef8974ad3f542a1ade4a3685f4042c4" candidate_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d230d00b0cc496fba0bbfd0b423855c"
I20260808 02:12:49.183847 20848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef8974ad3f542a1ade4a3685f4042c4" candidate_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "686779813126482b87ffb739fe669426"
I20260808 02:12:49.183957 20700 raft_consensus.cc:3037] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:49.184005 20848 raft_consensus.cc:3037] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:49.185294 20700 raft_consensus.cc:2454] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5c6eb4464af44eaa920c42f3db6fb48 in term 1.
I20260808 02:12:49.185845 20480 leader_election.cc:302] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d230d00b0cc496fba0bbfd0b423855c, c5c6eb4464af44eaa920c42f3db6fb48; no voters:
I20260808 02:12:49.185936 20848 raft_consensus.cc:2454] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5c6eb4464af44eaa920c42f3db6fb48 in term 1.
I20260808 02:12:49.186050 20930 raft_consensus.cc:2781] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:49.186357 20930 raft_consensus.cc:686] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 LEADER]: Becoming Leader. State: Replica: c5c6eb4464af44eaa920c42f3db6fb48, State: Running, Role: LEADER
I20260808 02:12:49.186765 20930 consensus_queue.cc:237] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } }
I20260808 02:12:49.190415 20382 catalog_manager.cc:5495] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5c6eb4464af44eaa920c42f3db6fb48 (127.18.184.129). New cstate: current_term: 1 leader_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } health_report { overall_health: HEALTHY } } }
I20260808 02:12:49.213122 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.209449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51246 (local address 127.18.184.129:34377)
0808 02:12:49.209603 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.209607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.209623 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:12:49.209793 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.209795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.209853 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.209947 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:49.210184 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.210685 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.211608 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.211870 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.211956 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.212265 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:49.212762 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:49.212853 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.212953 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.213006 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:12:49.219310 20848 raft_consensus.cc:1261] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426 [term 1 FOLLOWER]: Refusing update from remote peer c5c6eb4464af44eaa920c42f3db6fb48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:49.219368 20700 raft_consensus.cc:1261] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c [term 1 FOLLOWER]: Refusing update from remote peer c5c6eb4464af44eaa920c42f3db6fb48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:49.220091 20930 consensus_queue.cc:1035] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:49.220361 20934 consensus_queue.cc:1035] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:49.231009 20939 mvcc.cc:205] Tried to move back new op lower bound from 7316091573114220544 to 7316091572992180224. Current Snapshot: MvccSnapshot[applied={T|T < 7316091573114220544}]
I20260808 02:12:49.235198 20937 mvcc.cc:205] Tried to move back new op lower bound from 7316091573114220544 to 7316091572992180224. Current Snapshot: MvccSnapshot[applied={T|T < 7316091573114220544 or (T in {7316091573114220544})}]
I20260808 02:12:49.242713 20382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:49.242914 20382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:49.246062 20382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286: Sending DeleteTablet for 3 replicas of tablet eef8974ad3f542a1ade4a3685f4042c4
I20260808 02:12:49.246978 20829 tablet_service.cc:1514] Processing DeleteTablet for tablet eef8974ad3f542a1ade4a3685f4042c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:49 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:38084
I20260808 02:12:49.247143 20678 tablet_service.cc:1514] Processing DeleteTablet for tablet eef8974ad3f542a1ade4a3685f4042c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:49 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:34444
I20260808 02:12:49.247177 20526 tablet_service.cc:1514] Processing DeleteTablet for tablet eef8974ad3f542a1ade4a3685f4042c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:49 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:51242
I20260808 02:12:49.247561 20947 tablet_replica.cc:330] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426: stopping tablet replica
I20260808 02:12:49.247741 20948 tablet_replica.cc:330] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48: stopping tablet replica
I20260808 02:12:49.247910 20947 raft_consensus.cc:2229] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:49.248165 20948 raft_consensus.cc:2229] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:12:49.248241 20947 raft_consensus.cc:2258] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:49.248724 20948 raft_consensus.cc:2258] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:49.248726 20382 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:12:49.249281 20382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:49.249879 20947 ts_tablet_manager.cc:1905] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:49.250635 20948 ts_tablet_manager.cc:1905] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:49.253376 20947 ts_tablet_manager.cc:1918] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:12:49.253476 20947 log.cc:1196] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/eef8974ad3f542a1ade4a3685f4042c4
I20260808 02:12:49.253960 20947 ts_tablet_manager.cc:1939] T eef8974ad3f542a1ade4a3685f4042c4 P 686779813126482b87ffb739fe669426: Deleting consensus metadata
I20260808 02:12:49.254616 20948 ts_tablet_manager.cc:1918] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:12:49.254732 20948 log.cc:1196] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/wal/wals/eef8974ad3f542a1ade4a3685f4042c4
I20260808 02:12:49.254918 20829 tablet_service.cc:1467] Processing CreateTablet for tablet 5ca1a589a2b94c82b8b36d48c1912f0d (DEFAULT_TABLE table=test-table [id=f1a948f8372a49a6a5f769c9c48e2fe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:49.254918 20526 tablet_service.cc:1467] Processing CreateTablet for tablet 5ca1a589a2b94c82b8b36d48c1912f0d (DEFAULT_TABLE table=test-table [id=f1a948f8372a49a6a5f769c9c48e2fe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:49.255125 20948 ts_tablet_manager.cc:1939] T eef8974ad3f542a1ade4a3685f4042c4 P c5c6eb4464af44eaa920c42f3db6fb48: Deleting consensus metadata
I20260808 02:12:49.255204 20526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ca1a589a2b94c82b8b36d48c1912f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:49.255204 20829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ca1a589a2b94c82b8b36d48c1912f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:49.255786 20678 tablet_service.cc:1467] Processing CreateTablet for tablet 5ca1a589a2b94c82b8b36d48c1912f0d (DEFAULT_TABLE table=test-table [id=f1a948f8372a49a6a5f769c9c48e2fe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:49.256110 20678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ca1a589a2b94c82b8b36d48c1912f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:49.256650 20369 catalog_manager.cc:4841] TS c5c6eb4464af44eaa920c42f3db6fb48 (127.18.184.129:34377): tablet eef8974ad3f542a1ade4a3685f4042c4 (table test-table [id=8e0b4d578457480ca0271d87cdf7b994]) successfully deleted
I20260808 02:12:49.257496 20922 tablet_bootstrap.cc:492] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48: Bootstrap starting.
I20260808 02:12:49.257498 20924 tablet_bootstrap.cc:492] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426: Bootstrap starting.
I20260808 02:12:49.258543 20922 tablet_bootstrap.cc:654] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:49.258594 20924 tablet_bootstrap.cc:654] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:49.258673 20923 tablet_bootstrap.cc:492] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c: Bootstrap starting.
I20260808 02:12:49.259779 20923 tablet_bootstrap.cc:654] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:49.260001 20369 catalog_manager.cc:4841] TS 686779813126482b87ffb739fe669426 (127.18.184.131:33885): tablet eef8974ad3f542a1ade4a3685f4042c4 (table test-table [id=8e0b4d578457480ca0271d87cdf7b994]) successfully deleted
I20260808 02:12:49.260907 20922 tablet_bootstrap.cc:492] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48: No bootstrap required, opened a new log
I20260808 02:12:49.260998 20922 ts_tablet_manager.cc:1397] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 02:12:49.261222 20923 tablet_bootstrap.cc:492] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c: No bootstrap required, opened a new log
I20260808 02:12:49.261327 20923 ts_tablet_manager.cc:1397] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:12:49.261477 20922 raft_consensus.cc:348] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.261616 20922 raft_consensus.cc:374] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:49.261659 20922 raft_consensus.cc:729] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c6eb4464af44eaa920c42f3db6fb48, State: Initialized, Role: FOLLOWER
I20260808 02:12:49.261659 20950 tablet_replica.cc:330] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c: stopping tablet replica
I20260808 02:12:49.261793 20922 consensus_queue.cc:260] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.261901 20923 raft_consensus.cc:348] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.262039 20923 raft_consensus.cc:374] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:49.262061 20950 raft_consensus.cc:2229] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:49.262084 20923 raft_consensus.cc:729] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d230d00b0cc496fba0bbfd0b423855c, State: Initialized, Role: FOLLOWER
I20260808 02:12:49.262282 20923 consensus_queue.cc:260] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.262396 20950 raft_consensus.cc:2258] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:49.262697 20923 ts_tablet_manager.cc:1428] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:49.263216 20924 tablet_bootstrap.cc:492] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426: No bootstrap required, opened a new log
I20260808 02:12:49.263314 20924 ts_tablet_manager.cc:1397] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 02:12:49.263748 20924 raft_consensus.cc:348] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.263880 20924 raft_consensus.cc:374] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:49.263913 20924 raft_consensus.cc:729] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 686779813126482b87ffb739fe669426, State: Initialized, Role: FOLLOWER
I20260808 02:12:49.264035 20924 consensus_queue.cc:260] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.264274 20924 ts_tablet_manager.cc:1428] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:49.265205 20922 ts_tablet_manager.cc:1428] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260808 02:12:49.274677 20950 ts_tablet_manager.cc:1905] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:49.277848 20950 ts_tablet_manager.cc:1918] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:12:49.277992 20950 log.cc:1196] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/eef8974ad3f542a1ade4a3685f4042c4
I20260808 02:12:49.278345 20950 ts_tablet_manager.cc:1939] T eef8974ad3f542a1ade4a3685f4042c4 P 5d230d00b0cc496fba0bbfd0b423855c: Deleting consensus metadata
I20260808 02:12:49.279645 20369 catalog_manager.cc:4841] TS 5d230d00b0cc496fba0bbfd0b423855c (127.18.184.130:41927): tablet eef8974ad3f542a1ade4a3685f4042c4 (table test-table [id=8e0b4d578457480ca0271d87cdf7b994]) successfully deleted
W20260808 02:12:49.291174 20899 tablet.cc:2369] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:12:49.291711 20598 tablet.cc:2369] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:12:49.292661 20751 tablet.cc:2369] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:12:49.376397 20930 raft_consensus.cc:482] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:49.376571 20930 raft_consensus.cc:504] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.377076 20930 leader_election.cc:288] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d230d00b0cc496fba0bbfd0b423855c (127.18.184.130:41927), 686779813126482b87ffb739fe669426 (127.18.184.131:33885)
I20260808 02:12:49.377641 20700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ca1a589a2b94c82b8b36d48c1912f0d" candidate_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d230d00b0cc496fba0bbfd0b423855c" is_pre_election: true
I20260808 02:12:49.377825 20700 raft_consensus.cc:2454] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5c6eb4464af44eaa920c42f3db6fb48 in term 0.
I20260808 02:12:49.377797 20848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ca1a589a2b94c82b8b36d48c1912f0d" candidate_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "686779813126482b87ffb739fe669426" is_pre_election: true
I20260808 02:12:49.377941 20848 raft_consensus.cc:2454] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5c6eb4464af44eaa920c42f3db6fb48 in term 0.
I20260808 02:12:49.378211 20480 leader_election.cc:302] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d230d00b0cc496fba0bbfd0b423855c, c5c6eb4464af44eaa920c42f3db6fb48; no voters:
I20260808 02:12:49.378414 20930 raft_consensus.cc:2781] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:49.378491 20930 raft_consensus.cc:482] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:49.378540 20930 raft_consensus.cc:3037] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:49.379905 20930 raft_consensus.cc:504] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.380286 20930 leader_election.cc:288] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [CANDIDATE]: Term 1 election: Requested vote from peers 5d230d00b0cc496fba0bbfd0b423855c (127.18.184.130:41927), 686779813126482b87ffb739fe669426 (127.18.184.131:33885)
I20260808 02:12:49.380821 20700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ca1a589a2b94c82b8b36d48c1912f0d" candidate_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d230d00b0cc496fba0bbfd0b423855c"
I20260808 02:12:49.380852 20848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ca1a589a2b94c82b8b36d48c1912f0d" candidate_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "686779813126482b87ffb739fe669426"
I20260808 02:12:49.380980 20848 raft_consensus.cc:3037] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:49.380980 20700 raft_consensus.cc:3037] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:49.382155 20700 raft_consensus.cc:2454] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5c6eb4464af44eaa920c42f3db6fb48 in term 1.
I20260808 02:12:49.382246 20848 raft_consensus.cc:2454] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5c6eb4464af44eaa920c42f3db6fb48 in term 1.
I20260808 02:12:49.382586 20480 leader_election.cc:302] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 686779813126482b87ffb739fe669426, c5c6eb4464af44eaa920c42f3db6fb48; no voters:
I20260808 02:12:49.382817 20930 raft_consensus.cc:2781] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:49.382925 20930 raft_consensus.cc:686] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 LEADER]: Becoming Leader. State: Replica: c5c6eb4464af44eaa920c42f3db6fb48, State: Running, Role: LEADER
I20260808 02:12:49.383083 20930 consensus_queue.cc:237] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.385231 20380 catalog_manager.cc:5495] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5c6eb4464af44eaa920c42f3db6fb48 (127.18.184.129). New cstate: current_term: 1 leader_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } health_report { overall_health: UNKNOWN } } }
Aug 08 02:12:49 dist-test-slave-9gcw krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155168, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:49.445506 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.435409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57524 (local address 127.18.184.190:37395)
0808 02:12:49.435568 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.435572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.435793 (+ 221us) server_negotiation.cc:407] Connection header received
0808 02:12:49.435874 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.435876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.435922 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.436003 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:49.436763 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.437353 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.438283 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.438525 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.438613 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.441507 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:49.441510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:49.441511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:49.441536 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:49.443075 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:49.443840 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:49.443843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:49.443844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:49.443978 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:49.444295 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:49.444301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:49.444304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:49.444514 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:49.444592 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.445081 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.445361 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":72}
I20260808 02:12:49.445511 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.435305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:37395 (local address 127.0.0.1:57524)
0808 02:12:49.435669 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:49.435683 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:49.435797 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:49.436054 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:49.436057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:49.436524 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:49.436630 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.436637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.437505 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.437509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.438123 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.438132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.438644 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.438647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.438740 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.439397 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:49.439412 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:49.441328 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:49.443230 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:49.443240 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:49.443246 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:49.443517 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:49.444078 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:49.444081 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:49.444082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:49.444180 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:49.444646 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:49.444653 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:49.444869 (+ 216us) client_negotiation.cc:769] Sending connection context
0808 02:12:49.445003 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 02:12:49.445300 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":56,"thread_start_us":158,"threads_started":1}
I20260808 02:12:49.450392 20380 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:57524:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:12:49.451297 20380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:49.456985 20678 tablet_service.cc:1467] Processing CreateTablet for tablet e7f071b836f843cdbe260cb20c7c1824 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=779f41f0dbb849fdac4bba3e635798cb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:12:49.456986 20526 tablet_service.cc:1467] Processing CreateTablet for tablet e7f071b836f843cdbe260cb20c7c1824 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=779f41f0dbb849fdac4bba3e635798cb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:12:49.457068 20829 tablet_service.cc:1467] Processing CreateTablet for tablet e7f071b836f843cdbe260cb20c7c1824 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=779f41f0dbb849fdac4bba3e635798cb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:12:49.457455 20526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f071b836f843cdbe260cb20c7c1824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:49.457455 20678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f071b836f843cdbe260cb20c7c1824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:49.457584 20829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f071b836f843cdbe260cb20c7c1824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:49.463814 20923 tablet_bootstrap.cc:492] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c: Bootstrap starting.
I20260808 02:12:49.463835 20924 tablet_bootstrap.cc:492] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426: Bootstrap starting.
I20260808 02:12:49.463848 20922 tablet_bootstrap.cc:492] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: Bootstrap starting.
I20260808 02:12:49.464713 20923 tablet_bootstrap.cc:654] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:49.464808 20924 tablet_bootstrap.cc:654] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:49.464803 20922 tablet_bootstrap.cc:654] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:49.466457 20924 tablet_bootstrap.cc:492] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426: No bootstrap required, opened a new log
I20260808 02:12:49.466558 20924 ts_tablet_manager.cc:1397] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 02:12:49.466681 20923 tablet_bootstrap.cc:492] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c: No bootstrap required, opened a new log
I20260808 02:12:49.466770 20922 tablet_bootstrap.cc:492] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: No bootstrap required, opened a new log
I20260808 02:12:49.466789 20923 ts_tablet_manager.cc:1397] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:12:49.466872 20922 ts_tablet_manager.cc:1397] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 02:12:49.467078 20924 raft_consensus.cc:348] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.467216 20924 raft_consensus.cc:374] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:49.467252 20924 raft_consensus.cc:729] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 686779813126482b87ffb739fe669426, State: Initialized, Role: FOLLOWER
I20260808 02:12:49.467263 20923 raft_consensus.cc:348] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.467291 20922 raft_consensus.cc:348] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.467387 20923 raft_consensus.cc:374] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:49.467397 20922 raft_consensus.cc:374] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:49.467430 20922 raft_consensus.cc:729] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c6eb4464af44eaa920c42f3db6fb48, State: Initialized, Role: FOLLOWER
I20260808 02:12:49.467427 20923 raft_consensus.cc:729] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d230d00b0cc496fba0bbfd0b423855c, State: Initialized, Role: FOLLOWER
I20260808 02:12:49.467422 20924 consensus_queue.cc:260] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.467561 20922 consensus_queue.cc:260] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.467559 20923 consensus_queue.cc:260] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.467662 20930 raft_consensus.cc:482] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:49.467728 20930 raft_consensus.cc:504] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.467844 20922 ts_tablet_manager.cc:1428] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:49.467851 20923 ts_tablet_manager.cc:1428] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:49.468112 20930 leader_election.cc:288] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d230d00b0cc496fba0bbfd0b423855c (127.18.184.130:41927), 686779813126482b87ffb739fe669426 (127.18.184.131:33885)
I20260808 02:12:49.468127 20944 tablet_replica.cc:439] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } }
I20260808 02:12:49.468317 20944 tablet_replica.cc:442] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:49.468525 20928 tablet_replica.cc:439] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } }
I20260808 02:12:49.468681 20928 tablet_replica.cc:442] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:49.468842 20700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f071b836f843cdbe260cb20c7c1824" candidate_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d230d00b0cc496fba0bbfd0b423855c" is_pre_election: true
I20260808 02:12:49.468961 20700 raft_consensus.cc:2454] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5c6eb4464af44eaa920c42f3db6fb48 in term 0.
I20260808 02:12:49.469125 20848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f071b836f843cdbe260cb20c7c1824" candidate_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "686779813126482b87ffb739fe669426" is_pre_election: true
I20260808 02:12:49.469277 20924 ts_tablet_manager.cc:1428] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 02:12:49.469271 20848 raft_consensus.cc:2454] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5c6eb4464af44eaa920c42f3db6fb48 in term 0.
I20260808 02:12:49.469794 20480 leader_election.cc:302] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d230d00b0cc496fba0bbfd0b423855c, c5c6eb4464af44eaa920c42f3db6fb48; no voters:
I20260808 02:12:49.469902 20931 tablet_replica.cc:439] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } }
I20260808 02:12:49.470044 20931 tablet_replica.cc:442] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:49.470119 20944 raft_consensus.cc:2781] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:49.470216 20944 raft_consensus.cc:482] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:49.470300 20944 raft_consensus.cc:3037] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:49.471885 20944 raft_consensus.cc:504] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.472256 20944 leader_election.cc:288] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [CANDIDATE]: Term 1 election: Requested vote from peers 5d230d00b0cc496fba0bbfd0b423855c (127.18.184.130:41927), 686779813126482b87ffb739fe669426 (127.18.184.131:33885)
I20260808 02:12:49.472720 20700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f071b836f843cdbe260cb20c7c1824" candidate_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d230d00b0cc496fba0bbfd0b423855c"
I20260808 02:12:49.472769 20848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f071b836f843cdbe260cb20c7c1824" candidate_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "686779813126482b87ffb739fe669426"
I20260808 02:12:49.472846 20700 raft_consensus.cc:3037] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:49.472863 20848 raft_consensus.cc:3037] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:49.473989 20700 raft_consensus.cc:2454] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5c6eb4464af44eaa920c42f3db6fb48 in term 1.
I20260808 02:12:49.473989 20848 raft_consensus.cc:2454] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5c6eb4464af44eaa920c42f3db6fb48 in term 1.
I20260808 02:12:49.474260 20480 leader_election.cc:302] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d230d00b0cc496fba0bbfd0b423855c, c5c6eb4464af44eaa920c42f3db6fb48; no voters:
I20260808 02:12:49.474664 20944 raft_consensus.cc:2781] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:49.474852 20944 raft_consensus.cc:686] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 LEADER]: Becoming Leader. State: Replica: c5c6eb4464af44eaa920c42f3db6fb48, State: Running, Role: LEADER
I20260808 02:12:49.475013 20944 consensus_queue.cc:237] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } }
I20260808 02:12:49.476749 20930 tablet_replica.cc:439] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: TxnStatusTablet state changed. Reason: New leader c5c6eb4464af44eaa920c42f3db6fb48. Latest consensus state: current_term: 1 leader_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } }
I20260808 02:12:49.476878 20930 tablet_replica.cc:442] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:12:49.477257 20966 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:12:49.477450 20966 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:12:49.478052 20382 catalog_manager.cc:5495] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5c6eb4464af44eaa920c42f3db6fb48 (127.18.184.129). New cstate: current_term: 1 leader_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } health_report { overall_health: UNKNOWN } } }
I20260808 02:12:49.497946 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.494288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.18.184.129:34377)
0808 02:12:49.494451 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.494454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.494467 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:12:49.494515 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.494517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.494566 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.494659 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:49.495022 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.495689 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.496614 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.496912 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.497042 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.497404 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:49.497495 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:49.497583 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.497728 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.497785 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:12:49.497967 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.493785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:34377 (local address 127.0.0.1:51254)
0808 02:12:49.494005 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:49.494020 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:49.494114 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:49.494790 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:49.494793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:49.494800 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:49.494894 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.494899 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.495810 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.495815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.496481 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.496490 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.497068 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.497073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.497196 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.497246 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:49.497587 (+ 341us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:49.497607 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:12:49.497711 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 02:12:49.497806 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 02:12:49.501358 20848 raft_consensus.cc:1261] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426 [term 1 FOLLOWER]: Refusing update from remote peer c5c6eb4464af44eaa920c42f3db6fb48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:49.501380 20700 raft_consensus.cc:1261] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c [term 1 FOLLOWER]: Refusing update from remote peer c5c6eb4464af44eaa920c42f3db6fb48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:49.501935 20930 consensus_queue.cc:1035] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:49.502224 20944 consensus_queue.cc:1035] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:49.503729 20931 tablet_replica.cc:439] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426: TxnStatusTablet state changed. Reason: New leader c5c6eb4464af44eaa920c42f3db6fb48. Latest consensus state: current_term: 1 leader_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } }
I20260808 02:12:49.503854 20931 tablet_replica.cc:442] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:49.504379 20928 tablet_replica.cc:439] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c: TxnStatusTablet state changed. Reason: New leader c5c6eb4464af44eaa920c42f3db6fb48. Latest consensus state: current_term: 1 leader_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } }
I20260808 02:12:49.504521 20928 tablet_replica.cc:442] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:49.505987 20934 tablet_replica.cc:439] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } }
I20260808 02:12:49.506115 20934 tablet_replica.cc:442] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:12:49.506430 20934 tablet_replica.cc:439] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } }
I20260808 02:12:49.506523 20934 tablet_replica.cc:442] T e7f071b836f843cdbe260cb20c7c1824 P c5c6eb4464af44eaa920c42f3db6fb48: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:12:49.507838 20931 tablet_replica.cc:439] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } }
I20260808 02:12:49.507956 20931 tablet_replica.cc:442] T e7f071b836f843cdbe260cb20c7c1824 P 686779813126482b87ffb739fe669426: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:49.508598 20938 mvcc.cc:205] Tried to move back new op lower bound from 7316091574272200704 to 7316091574171422720. Current Snapshot: MvccSnapshot[applied={T|T < 7316091574272200704 or (T in {7316091574272200704})}]
I20260808 02:12:49.509706 20928 tablet_replica.cc:439] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c6eb4464af44eaa920c42f3db6fb48" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 34377 } } peers { permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 } } peers { permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 } } }
I20260808 02:12:49.509825 20928 tablet_replica.cc:442] T e7f071b836f843cdbe260cb20c7c1824 P 5d230d00b0cc496fba0bbfd0b423855c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:49.525882 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.522144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:58347 (local address 127.18.184.129:34377)
0808 02:12:49.522284 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.522289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.522459 (+ 170us) server_negotiation.cc:407] Connection header received
0808 02:12:49.522588 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.522590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.522637 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.522714 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:49.522988 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.523501 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.524752 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.524979 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.525071 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.525426 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:49.525498 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:49.525569 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.525672 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.525734 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":61}
I20260808 02:12:49.525944 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.522074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:34377 (local address 127.18.184.129:58347)
0808 02:12:49.522364 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:49.522380 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:49.522494 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:49.522748 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:49.522752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:49.522759 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:49.522886 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.522891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.523609 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.523612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.524597 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:49.524608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.525104 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.525107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:49.525228 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.525282 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:49.525590 (+ 308us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:49.525597 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:12:49.525666 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:12:49.525776 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":22,"thread_start_us":111,"threads_started":1}
I20260808 02:12:49.533464 20848 raft_consensus.cc:1261] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426 [term 1 FOLLOWER]: Refusing update from remote peer c5c6eb4464af44eaa920c42f3db6fb48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:49.533923 20700 raft_consensus.cc:1261] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c [term 1 FOLLOWER]: Refusing update from remote peer c5c6eb4464af44eaa920c42f3db6fb48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:49.533921 20930 consensus_queue.cc:1035] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "686779813126482b87ffb739fe669426" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 33885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:49.534265 20930 consensus_queue.cc:1035] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d230d00b0cc496fba0bbfd0b423855c" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:49.584308 20382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:49.584476 20382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:49.586751 20382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ab5a034dbee4188927452d1a8b22286: Sending DeleteTablet for 3 replicas of tablet 5ca1a589a2b94c82b8b36d48c1912f0d
I20260808 02:12:49.587397 20526 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ca1a589a2b94c82b8b36d48c1912f0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:49 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:51242
I20260808 02:12:49.587522 20829 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ca1a589a2b94c82b8b36d48c1912f0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:49 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:38084
I20260808 02:12:49.587621 20948 tablet_replica.cc:330] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48: stopping tablet replica
I20260808 02:12:49.587668 20678 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ca1a589a2b94c82b8b36d48c1912f0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:49 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:34444
I20260808 02:12:49.587754 20948 raft_consensus.cc:2229] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:12:49.587766 20947 tablet_replica.cc:330] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426: stopping tablet replica
I20260808 02:12:49.587844 20950 tablet_replica.cc:330] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c: stopping tablet replica
I20260808 02:12:49.587881 20947 raft_consensus.cc:2229] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:49.587945 20950 raft_consensus.cc:2229] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:49.587993 20948 raft_consensus.cc:2258] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:49.588035 20947 raft_consensus.cc:2258] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:49.588084 20950 raft_consensus.cc:2258] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:49.589003 20948 ts_tablet_manager.cc:1905] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:49.589206 20950 ts_tablet_manager.cc:1905] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:49.592412 20948 ts_tablet_manager.cc:1918] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:49.592412 20950 ts_tablet_manager.cc:1918] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:49.592515 20950 log.cc:1196] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/5ca1a589a2b94c82b8b36d48c1912f0d
I20260808 02:12:49.592515 20948 log.cc:1196] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-0/wal/wals/5ca1a589a2b94c82b8b36d48c1912f0d
I20260808 02:12:49.592839 20948 ts_tablet_manager.cc:1939] T 5ca1a589a2b94c82b8b36d48c1912f0d P c5c6eb4464af44eaa920c42f3db6fb48: Deleting consensus metadata
I20260808 02:12:49.592839 20950 ts_tablet_manager.cc:1939] T 5ca1a589a2b94c82b8b36d48c1912f0d P 5d230d00b0cc496fba0bbfd0b423855c: Deleting consensus metadata
I20260808 02:12:49.593734 20369 catalog_manager.cc:4841] TS c5c6eb4464af44eaa920c42f3db6fb48 (127.18.184.129:34377): tablet 5ca1a589a2b94c82b8b36d48c1912f0d (table test-table [id=f1a948f8372a49a6a5f769c9c48e2fe1]) successfully deleted
I20260808 02:12:49.593961 20369 catalog_manager.cc:4841] TS 5d230d00b0cc496fba0bbfd0b423855c (127.18.184.130:41927): tablet 5ca1a589a2b94c82b8b36d48c1912f0d (table test-table [id=f1a948f8372a49a6a5f769c9c48e2fe1]) successfully deleted
I20260808 02:12:49.594200 20947 ts_tablet_manager.cc:1905] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:49.596833 20947 ts_tablet_manager.cc:1918] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:49.596905 20947 log.cc:1196] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/5ca1a589a2b94c82b8b36d48c1912f0d
I20260808 02:12:49.597322 20947 ts_tablet_manager.cc:1939] T 5ca1a589a2b94c82b8b36d48c1912f0d P 686779813126482b87ffb739fe669426: Deleting consensus metadata
I20260808 02:12:49.598212 20369 catalog_manager.cc:4841] TS 686779813126482b87ffb739fe669426 (127.18.184.131:33885): tablet 5ca1a589a2b94c82b8b36d48c1912f0d (table test-table [id=f1a948f8372a49a6a5f769c9c48e2fe1]) successfully deleted
Aug 08 02:12:49 dist-test-slave-9gcw krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.184.129@KRBTEST.COM
I20260808 02:12:49.607299 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.594578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51260 (local address 127.18.184.129:34377)
0808 02:12:49.594694 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.594699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.595010 (+ 311us) server_negotiation.cc:407] Connection header received
0808 02:12:49.595205 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.595208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.595258 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.595313 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:49.596353 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.597062 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.598611 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.598970 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.599105 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.602856 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:49.602867 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:49.602885 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:49.602917 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:49.605036 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:49.605558 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:49.605565 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:49.605567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:49.605668 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:49.606004 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:49.606008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:49.606009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:49.606328 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:49.606568 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.606923 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.607065 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 02:12:49.607782 20591 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51260
I20260808 02:12:49.618994 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:49.609849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57536 (local address 127.18.184.190:37395)
0808 02:12:49.609985 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:49.609989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:49.610043 (+ 54us) server_negotiation.cc:407] Connection header received
0808 02:12:49.610180 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:49.610183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:49.610235 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:49.610324 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:49.610965 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.611717 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.612604 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:49.612932 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:49.613040 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:49.614860 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:49.614864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:49.614867 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:49.614900 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:49.616734 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:49.617297 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:49.617301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:49.617303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:49.617405 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:49.617714 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:49.617718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:49.617720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:49.618040 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:49.618201 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:49.618606 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 02:12:49.618752 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 02:12:49.619522 20382 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57536
I20260808 02:12:49.621114 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 20460
I20260808 02:12:49.632691 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 20606
I20260808 02:12:49.640340 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 20759
I20260808 02:12:49.648713 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 20348
2026-08-08T02:12:49Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2862 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:12:49 dist-test-slave-9gcw krb5kdc[20982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:12:49 dist-test-slave-9gcw krb5kdc[20982](info): set up 2 sockets
Aug 08 02:12:49 dist-test-slave-9gcw krb5kdc[20982](info): commencing operation
krb5kdc: starting...
W20260808 02:12:50.798583 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:12:50 dist-test-slave-9gcw krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:12:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:12:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:12:50.945431 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:42871
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:42871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:51.062500 20996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:51.062847 20996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:51.062932 20996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:51.067072 20996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:12:51.067163 20996 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:12:51.067194 20996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:51.067248 20996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:51.067292 20996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:12:51.067329 20996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:12:51.072818 20996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:42871
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:42871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:51.074069 20996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:51.075150 20996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:51.084156 21004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:51.084229 21001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:51.084419 20996 server_base.cc:1034] running on GCE node
W20260808 02:12:51.084513 21002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:51.085060 20996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:51.086247 20996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:51.087430 20996 hybrid_clock.cc:648] HybridClock initialized: now 1786155171087417 us; error 26 us; skew 500 ppm
Aug 08 02:12:51 dist-test-slave-9gcw krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:51.090551 20996 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:12:51.091547 20996 webserver.cc:460] Webserver started at http://127.18.184.190:37467/ using document root <none> and password file <none>
I20260808 02:12:51.092171 20996 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:51.092242 20996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:51.092470 20996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:51.094677 20996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "c31d9845d8884302b33caeca5515f0b6"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-9gcw"
server_key: "ccf5098ad4ab82bdcf1c5b11d5839d82"
server_key_iv: "1f8436b45e8ae7c4327a4c0f7cd2b14a"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.095278 20996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "c31d9845d8884302b33caeca5515f0b6"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-9gcw"
server_key: "ccf5098ad4ab82bdcf1c5b11d5839d82"
server_key_iv: "1f8436b45e8ae7c4327a4c0f7cd2b14a"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.099417 20996 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:12:51.102298 21011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:51.103958 20996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:12:51.104137 20996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "c31d9845d8884302b33caeca5515f0b6"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-9gcw"
server_key: "ccf5098ad4ab82bdcf1c5b11d5839d82"
server_key_iv: "1f8436b45e8ae7c4327a4c0f7cd2b14a"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.104280 20996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:51.154860 20996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:51.158869 20996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:51.159118 20996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:51.169098 20996 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:42871
I20260808 02:12:51.169118 21079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:42871 every 8 connection(s)
I20260808 02:12:51.170331 20996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:12:51.173257 21080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:51.173960 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 20996
I20260808 02:12:51.174078 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:12:51.174402 19170 external_mini_cluster.cc:1428] Setting key e6df23a0fe81a897e536713bffa9b7a8
I20260808 02:12:51.180982 21080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6: Bootstrap starting.
I20260808 02:12:51.183943 21080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6: Neither blocks nor log segments found. Creating new log.
Aug 08 02:12:51 dist-test-slave-9gcw krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:51.184794 21080 log.cc:824] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:51.186851 21080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6: No bootstrap required, opened a new log
I20260808 02:12:51.189991 21080 raft_consensus.cc:348] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c31d9845d8884302b33caeca5515f0b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42871 } }
I20260808 02:12:51.190248 21080 raft_consensus.cc:374] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:51.190330 21080 raft_consensus.cc:729] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c31d9845d8884302b33caeca5515f0b6, State: Initialized, Role: FOLLOWER
I20260808 02:12:51.190532 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.175851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42320 (local address 127.18.184.190:42871)
0808 02:12:51.176286 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:51.176303 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:51.176341 (+ 38us) server_negotiation.cc:407] Connection header received
0808 02:12:51.177127 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:51.177171 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:51.177539 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:51.177868 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:51.178838 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.180048 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.181290 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.181782 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.181933 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.184957 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:51.184964 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:51.184982 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:51.185023 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:51.187485 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.187953 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.187959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.187971 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.188083 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.188377 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.188381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.188383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.188793 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:51.189019 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:51.189349 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 02:12:51.189640 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":143,"threads_started":1}
I20260808 02:12:51.190837 21080 consensus_queue.cc:260] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c31d9845d8884302b33caeca5515f0b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42871 } }
I20260808 02:12:51.190992 21080 raft_consensus.cc:388] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:12:51.191079 21080 raft_consensus.cc:482] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:12:51.191179 21080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:51.192121 21080 raft_consensus.cc:504] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c31d9845d8884302b33caeca5515f0b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42871 } }
I20260808 02:12:51.192469 21080 leader_election.cc:302] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c31d9845d8884302b33caeca5515f0b6; no voters:
I20260808 02:12:51.192853 21080 leader_election.cc:288] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:12:51.192973 21085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:51.193243 21085 raft_consensus.cc:686] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [term 1 LEADER]: Becoming Leader. State: Replica: c31d9845d8884302b33caeca5515f0b6, State: Running, Role: LEADER
I20260808 02:12:51.193550 21085 consensus_queue.cc:237] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c31d9845d8884302b33caeca5515f0b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42871 } }
I20260808 02:12:51.194223 21080 sys_catalog.cc:564] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:12:51.195399 21086 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c31d9845d8884302b33caeca5515f0b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c31d9845d8884302b33caeca5515f0b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42871 } } }
I20260808 02:12:51.195442 21088 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c31d9845d8884302b33caeca5515f0b6. Latest consensus state: current_term: 1 leader_uuid: "c31d9845d8884302b33caeca5515f0b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c31d9845d8884302b33caeca5515f0b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42871 } } }
I20260808 02:12:51.195544 21086 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:51.195554 21088 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:51.195953 21095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:12:51.199533 21095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:12:51.205756 21095 catalog_manager.cc:1347] Generated new cluster ID: a514555ce0d94a98b786c84d806b395c
I20260808 02:12:51.205852 21095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:12:51.215310 21095 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:12:51.216368 21095 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:12:51.222539 21095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6: Generated new TSK 0
I20260808 02:12:51.223315 21095 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:12:51.266950 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:42871
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:51.392917 21109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:51.393221 21109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:51.393275 21109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:51.397730 21109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:51.397799 21109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:51.397841 21109 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:12:51.397855 21109 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:12:51.397888 21109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:12:51.403301 21109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.184.190:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:51.404356 21109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:51.405450 21109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:51.415325 21115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:51.415326 21114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:51.415364 21117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:51.415621 21109 server_base.cc:1034] running on GCE node
I20260808 02:12:51.416127 21109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:51.416779 21109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:51.417989 21109 hybrid_clock.cc:648] HybridClock initialized: now 1786155171417956 us; error 44 us; skew 500 ppm
Aug 08 02:12:51 dist-test-slave-9gcw krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:51.421314 21109 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:12:51.422408 21109 webserver.cc:460] Webserver started at http://127.18.184.129:38267/ using document root <none> and password file <none>
I20260808 02:12:51.423035 21109 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:51.423139 21109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:51.423409 21109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:51.425913 21109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "51ad5049927c4e1d80556710a9ff1b1b"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-9gcw"
server_key: "5f6c19c9186ebf1283131dccf1755396"
server_key_iv: "625d34ee686dd1ffd0593e91670f5f82"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.426589 21109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "51ad5049927c4e1d80556710a9ff1b1b"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-9gcw"
server_key: "5f6c19c9186ebf1283131dccf1755396"
server_key_iv: "625d34ee686dd1ffd0593e91670f5f82"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.431147 21109 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:12:51.434237 21124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:51.435695 21109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:12:51.435835 21109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "51ad5049927c4e1d80556710a9ff1b1b"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-9gcw"
server_key: "5f6c19c9186ebf1283131dccf1755396"
server_key_iv: "625d34ee686dd1ffd0593e91670f5f82"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.435966 21109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:51.459481 21109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:51.462944 21109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:51.463191 21109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:51.464887 21109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:51.464965 21109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:51.465036 21109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:51.465080 21109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:12:51 dist-test-slave-9gcw krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:51.476698 21109 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:44725
I20260808 02:12:51.476800 21246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:44725 every 8 connection(s)
I20260808 02:12:51.477779 21109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:12:51.481315 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.465767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:50351 (local address 127.18.184.190:42871)
0808 02:12:51.465967 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:51.465972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:51.466858 (+ 886us) server_negotiation.cc:407] Connection header received
0808 02:12:51.467826 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:51.467830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:51.467892 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:51.467979 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:51.469721 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.470652 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.472634 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.473041 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.473213 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.476657 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:51.476662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:51.476665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:51.476701 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:51.478768 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.479407 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.479411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.479413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.479515 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.479925 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.479929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.479931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.480258 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:51.480383 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:51.480937 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 02:12:51.481089 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 02:12:51.482241 21133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.466181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:42871 (local address 127.18.184.129:50351)
0808 02:12:51.466660 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:51.466722 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:51.467594 (+ 872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:51.468323 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:51.468330 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:51.468856 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:51.469527 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:51.469548 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.470834 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.470841 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:51.472480 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:51.472498 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.473256 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.473262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:51.473466 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.474250 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:51.474277 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:51.476420 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:51.478923 (+ 2503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:51.478929 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:51.478946 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:51.479278 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:51.479621 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:51.479625 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:51.479628 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:51.479793 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:51.480379 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:51.480386 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:51.480674 (+ 288us) client_negotiation.cc:769] Sending connection context
0808 02:12:51.480889 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 02:12:51.481308 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":137,"threads_started":1}
I20260808 02:12:51.483654 21247 heartbeater.cc:347] Connected to a master server at 127.18.184.190:42871
I20260808 02:12:51.483812 21247 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:51.484380 21247 heartbeater.cc:511] Master 127.18.184.190:42871 requested a full tablet report, sending...
I20260808 02:12:51.486021 21027 ts_manager.cc:194] Registered new tserver with Master: 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725)
I20260808 02:12:51.486443 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 21109
I20260808 02:12:51.486598 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:12:51.486919 19170 external_mini_cluster.cc:1428] Setting key 754633e332449538a93937e6db5f79bc
I20260808 02:12:51.487607 21027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:50351
I20260808 02:12:51.494587 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.486103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:42871 (local address 127.18.184.129:40713)
0808 02:12:51.486496 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:51.486515 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:51.486628 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:51.487011 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:51.487015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:51.487238 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:51.487392 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:51.487400 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.488429 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.488431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:51.489024 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:51.489033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.489611 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.489615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:51.489709 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.490252 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:51.490298 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:51.490648 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:51.492759 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:51.492765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:51.492769 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:51.492992 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:51.493353 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:51.493356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:51.493358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:51.493452 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:51.493866 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:51.493869 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:51.494040 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 02:12:51.494183 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 02:12:51.494363 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":121,"threads_started":1}
I20260808 02:12:51.494580 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.486194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:40713 (local address 127.18.184.190:42871)
0808 02:12:51.486325 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:51.486329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:51.486582 (+ 253us) server_negotiation.cc:407] Connection header received
0808 02:12:51.486824 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:51.486827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:51.486875 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:51.486971 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:51.487585 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.488294 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.489207 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.489470 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.489602 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.490797 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:51.490801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:51.490804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:51.490835 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:51.492661 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.493102 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.493106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.493108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.493238 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.493549 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.493553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.493555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.493765 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:51.493881 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:51.494269 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 02:12:51.494397 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:12:51.521554 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:42871
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:51.640501 21257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:51.640779 21257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:51.640828 21257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:51.644884 21257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:51.644951 21257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:51.644992 21257 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:12:51.645007 21257 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:12:51.645038 21257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:12:51.650556 21257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.184.190:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:51.651651 21257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:51.652647 21257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:51.662825 21263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:51.662827 21266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:51.662887 21262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:51.663249 21257 server_base.cc:1034] running on GCE node
I20260808 02:12:51.663625 21257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:51.664265 21257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:51.665428 21257 hybrid_clock.cc:648] HybridClock initialized: now 1786155171665393 us; error 48 us; skew 500 ppm
Aug 08 02:12:51 dist-test-slave-9gcw krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:51.668421 21257 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:12:51.669462 21257 webserver.cc:460] Webserver started at http://127.18.184.130:36359/ using document root <none> and password file <none>
I20260808 02:12:51.670046 21257 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:51.670109 21257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:51.670342 21257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:51.672410 21257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "2fcf1968955e4146a69443b2105e3126"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-9gcw"
server_key: "8a567519c388fb5e572dd6a62db9bca0"
server_key_iv: "5f1bc55cebae9898757ffcad36e7b3a6"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.673007 21257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "2fcf1968955e4146a69443b2105e3126"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-9gcw"
server_key: "8a567519c388fb5e572dd6a62db9bca0"
server_key_iv: "5f1bc55cebae9898757ffcad36e7b3a6"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.677028 21257 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 02:12:51.679540 21273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:51.681185 21257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:12:51.681331 21257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "2fcf1968955e4146a69443b2105e3126"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-9gcw"
server_key: "8a567519c388fb5e572dd6a62db9bca0"
server_key_iv: "5f1bc55cebae9898757ffcad36e7b3a6"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.681444 21257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:51.701527 21257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:51.704969 21257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:51.705278 21257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:51.706868 21257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:51.706954 21257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:51.707036 21257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:51.707083 21257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:12:51 dist-test-slave-9gcw krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:51.719586 21257 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:40851
I20260808 02:12:51.719631 21395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:40851 every 8 connection(s)
I20260808 02:12:51.720841 21257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:12:51.723038 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.708224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:42871 (local address 127.18.184.130:49745)
0808 02:12:51.708735 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:51.708801 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:51.710056 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:51.710753 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:51.710763 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:51.711308 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:51.712048 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:51.712073 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.713278 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.713286 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:51.714870 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:51.714885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.715487 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.715491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:51.715673 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.716322 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:51.716345 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:51.717999 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:51.720099 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:51.720103 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:51.720114 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:51.720375 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:51.720737 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:51.720740 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:51.720742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:51.720877 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:51.721421 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:51.721428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:51.721721 (+ 293us) client_negotiation.cc:769] Sending connection context
0808 02:12:51.721947 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 02:12:51.722301 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":106,"threads_started":1}
I20260808 02:12:51.723691 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.707822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:49745 (local address 127.18.184.190:42871)
0808 02:12:51.708031 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:51.708040 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:51.708932 (+ 892us) server_negotiation.cc:407] Connection header received
0808 02:12:51.710385 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:51.710389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:51.710447 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:51.710552 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:51.712224 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.713112 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.715010 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.715273 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.715428 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.718180 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:51.718184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:51.718187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:51.718223 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:51.719977 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.720494 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.720498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.720500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.720621 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.721028 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.721031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.721032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.721305 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:51.721428 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:51.723454 (+ 2026us) server_negotiation.cc:299] Negotiation successful
0808 02:12:51.723575 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:12:51.724879 21396 heartbeater.cc:347] Connected to a master server at 127.18.184.190:42871
I20260808 02:12:51.725037 21396 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:51.725591 21396 heartbeater.cc:511] Master 127.18.184.190:42871 requested a full tablet report, sending...
I20260808 02:12:51.726730 21029 ts_manager.cc:194] Registered new tserver with Master: 2fcf1968955e4146a69443b2105e3126 (127.18.184.130:40851)
I20260808 02:12:51.727413 21029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:49745
I20260808 02:12:51.729809 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 21257
I20260808 02:12:51.729951 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:12:51.730242 19170 external_mini_cluster.cc:1428] Setting key a07c5f33e9a2d1747d07fc8c0793968a
I20260808 02:12:51.735044 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.727027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:47533 (local address 127.18.184.190:42871)
0808 02:12:51.727170 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:51.727177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:51.727384 (+ 207us) server_negotiation.cc:407] Connection header received
0808 02:12:51.727497 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:51.727500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:51.727568 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:51.727662 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:51.728244 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.728762 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.729697 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.729936 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.730037 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.731369 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:51.731375 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:51.731382 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:51.731415 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:51.732888 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.733481 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.733485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.733486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.733593 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.734051 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.734057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.734064 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.734303 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:51.734437 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:51.734750 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 02:12:51.734877 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:12:51.735062 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.726963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:42871 (local address 127.18.184.130:47533)
0808 02:12:51.727304 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:51.727315 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:51.727421 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:51.727800 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:51.727803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:51.727984 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:51.728121 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:51.728127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.728905 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.728908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:51.729554 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:51.729579 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.730055 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.730058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:51.730163 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.730754 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:51.730766 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:51.731211 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:51.733005 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:51.733011 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:51.733017 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:51.733355 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:51.733813 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:51.733817 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:51.733819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:51.733929 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:51.734427 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:51.734431 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:51.734544 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 02:12:51.734665 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 02:12:51.734837 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":2,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:12:51.764513 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:42871
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:51.889232 21405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:51.889575 21405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:51.889684 21405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:51.894002 21405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:51.894120 21405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:51.894205 21405 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:12:51.894260 21405 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:12:51.894332 21405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:12:51.900290 21405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.184.190:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:51.901502 21405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:51.902618 21405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:51.913080 21410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:51.913118 21411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:51.913204 21413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:51.913446 21405 server_base.cc:1034] running on GCE node
I20260808 02:12:51.913959 21405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:51.914580 21405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:51.915799 21405 hybrid_clock.cc:648] HybridClock initialized: now 1786155171915763 us; error 48 us; skew 500 ppm
Aug 08 02:12:51 dist-test-slave-9gcw krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:51.919004 21405 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:12:51.920035 21405 webserver.cc:460] Webserver started at http://127.18.184.131:38955/ using document root <none> and password file <none>
I20260808 02:12:51.920622 21405 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:51.920718 21405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:51.920974 21405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:51.923295 21405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "a17a3f03de0d4a68b2a8a3aa037398d7"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-9gcw"
server_key: "636105ac8f2d546ba3e5bf9f117fa9fb"
server_key_iv: "3ea8ea7d4924a21b24adc3de8e8bf216"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.923900 21405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "a17a3f03de0d4a68b2a8a3aa037398d7"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-9gcw"
server_key: "636105ac8f2d546ba3e5bf9f117fa9fb"
server_key_iv: "3ea8ea7d4924a21b24adc3de8e8bf216"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.927876 21405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 02:12:51.930562 21421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:51.931993 21405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:12:51.932153 21405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "a17a3f03de0d4a68b2a8a3aa037398d7"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-9gcw"
server_key: "636105ac8f2d546ba3e5bf9f117fa9fb"
server_key_iv: "3ea8ea7d4924a21b24adc3de8e8bf216"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.932291 21405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:51.950644 21405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:51.954120 21405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:51.954344 21405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:51.955916 21405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:51.955991 21405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:51.956066 21405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:51.956110 21405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:12:51 dist-test-slave-9gcw krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:51.967638 21405 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:45053
I20260808 02:12:51.967708 21543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:45053 every 8 connection(s)
I20260808 02:12:51.968899 21405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:12:51.971814 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.956914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:53041 (local address 127.18.184.190:42871)
0808 02:12:51.957087 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:51.957093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:51.958153 (+ 1060us) server_negotiation.cc:407] Connection header received
0808 02:12:51.959638 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:51.959645 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:51.959718 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:51.959849 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:51.961462 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.962386 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.964093 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.964455 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.964577 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.967574 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:51.967579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:51.967582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:51.967613 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:51.969624 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.970206 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.970210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.970212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.970325 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.970643 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.970646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.970647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.970916 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:51.971005 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:51.971504 (+ 499us) server_negotiation.cc:299] Negotiation successful
0808 02:12:51.971616 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:12:51.972599 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.957440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:42871 (local address 127.18.184.131:53041)
0808 02:12:51.957955 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:51.958016 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:51.959353 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:51.960098 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:51.960105 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:51.960588 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:51.961231 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:51.961259 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.962563 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.962573 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:51.963976 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:51.963987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.964632 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.964635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:51.964805 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.965536 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:51.965563 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:51.967345 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:51.969769 (+ 2424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:51.969775 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:51.969790 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:51.970074 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:51.970419 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:51.970421 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:51.970422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:51.970550 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:51.971012 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:51.971017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:51.971280 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 02:12:51.971433 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 02:12:51.971684 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":109,"threads_started":1}
I20260808 02:12:51.973179 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 21405
I20260808 02:12:51.973318 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:12:51.973629 19170 external_mini_cluster.cc:1428] Setting key 494b2f86a5077e4189cf95b53b5583d1
I20260808 02:12:51.974090 21544 heartbeater.cc:347] Connected to a master server at 127.18.184.190:42871
I20260808 02:12:51.974251 21544 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:51.974833 21544 heartbeater.cc:511] Master 127.18.184.190:42871 requested a full tablet report, sending...
I20260808 02:12:51.975924 21027 ts_manager.cc:194] Registered new tserver with Master: a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131:45053)
I20260808 02:12:51.976641 21027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:53041
I20260808 02:12:51.984189 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.976672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:42871 (local address 127.18.184.131:37179)
0808 02:12:51.977096 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:51.977108 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:51.977253 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:51.977491 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:51.977495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:51.977733 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:51.977859 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:51.977867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.978596 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.978599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:51.979211 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:51.979220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.979626 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.979629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:51.979790 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.980345 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:51.980366 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:51.980693 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:51.982460 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:51.982463 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:51.982464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:51.982750 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:51.983094 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:51.983096 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:51.983097 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:51.983176 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:51.983586 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:51.983588 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:51.983663 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 02:12:51.983822 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 02:12:51.984012 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":102,"threads_started":1}
I20260808 02:12:51.984203 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.976749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:37179 (local address 127.18.184.190:42871)
0808 02:12:51.976849 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:51.976854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:51.977212 (+ 358us) server_negotiation.cc:407] Connection header received
0808 02:12:51.977341 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:51.977343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:51.977381 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:51.977445 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:51.977976 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.978458 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.979319 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.979523 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.979608 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.980817 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:51.980820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:51.980822 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:51.980842 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:51.982354 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.982875 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.982880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.982883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.982997 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.983290 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.983293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.983294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.983490 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:51.983578 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:51.983933 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 02:12:51.984076 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 02:12:51.987974 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:12:51 dist-test-slave-9gcw krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:12:52 dist-test-slave-9gcw krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:52.017827 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.008172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42324 (local address 127.18.184.190:42871)
0808 02:12:52.008355 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.008368 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.008383 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:12:52.008448 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.008453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.008518 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.008620 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:52.009610 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.010211 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.011149 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.011403 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.011535 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.014309 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:52.014313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:52.014315 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:52.014341 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:52.015816 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.016393 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.016403 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.016404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.016487 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.016797 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.016802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.016804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.017031 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:52.017230 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.017497 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.017636 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:12:52.021353 21027 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:12:52.024653 21027 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:52.040678 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.033916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:40851 (local address 127.0.0.1:33394)
0808 02:12:52.034505 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.034568 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.034736 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.035780 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.035786 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.035811 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:52.036019 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.036026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.037330 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.037336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.038752 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.038763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.039970 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.039976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.040110 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.040153 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.040335 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.040524 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":1,"spinlock_wait_cycles":3712,"thread_start_us":97,"threads_started":1}
I20260808 02:12:52.040678 21562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.034032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33394 (local address 127.18.184.130:40851)
0808 02:12:52.035228 (+ 1196us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.035241 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.035265 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:12:52.035365 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.035371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.035602 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.035800 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:52.036194 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.037210 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.038908 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.039820 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.040003 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.040172 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.040370 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.040485 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1083,"thread_start_us":150,"threads_started":1}
I20260808 02:12:52.040966 21564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.034752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:45053 (local address 127.0.0.1:42044)
0808 02:12:52.035458 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.035471 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.035553 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.036137 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.036142 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.036162 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:52.036264 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.036269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.037896 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.037899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.039143 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.039155 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.040590 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.040593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.040685 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.040709 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.040780 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.040839 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":60,"thread_start_us":121,"threads_started":1}
I20260808 02:12:52.041438 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.035118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42044 (local address 127.18.184.131:45053)
0808 02:12:52.035639 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.035653 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.035677 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:12:52.035761 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.035767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.035973 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.036136 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:52.036403 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.037746 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.039300 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.040173 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.040386 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.040526 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.041182 (+ 656us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.041283 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":127,"threads_started":1}
I20260808 02:12:52.041795 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.034076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:44725 (local address 127.0.0.1:57584)
0808 02:12:52.034505 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.034568 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.034718 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.036168 (+ 1450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.036171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.036200 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:52.036361 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.036370 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.038035 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.038039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.039221 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.039248 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.040396 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.040400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.040502 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.040525 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.040581 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.040655 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":48,"spinlock_wait_cycles":5248,"thread_start_us":134,"threads_started":1}
I20260808 02:12:52.042083 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.035011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57584 (local address 127.18.184.129:44725)
0808 02:12:52.035631 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.035639 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.035678 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:12:52.035761 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.035767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.035973 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.036560 (+ 587us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:52.036722 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.037906 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.039401 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.040264 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.041660 (+ 1396us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.041781 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.041822 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.041949 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":116,"threads_started":1}
I20260808 02:12:52.044127 21325 tablet_service.cc:1467] Processing CreateTablet for tablet 0f351e82c57949e384d4e119ba53e527 (DEFAULT_TABLE table=test-table [id=9c3fa31e946240ae99206215fe643f20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:52.044502 21178 tablet_service.cc:1467] Processing CreateTablet for tablet 0f351e82c57949e384d4e119ba53e527 (DEFAULT_TABLE table=test-table [id=9c3fa31e946240ae99206215fe643f20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:52.045369 21325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f351e82c57949e384d4e119ba53e527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:52.045636 21178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f351e82c57949e384d4e119ba53e527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:52.047909 21468 tablet_service.cc:1467] Processing CreateTablet for tablet 0f351e82c57949e384d4e119ba53e527 (DEFAULT_TABLE table=test-table [id=9c3fa31e946240ae99206215fe643f20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:52.049181 21468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f351e82c57949e384d4e119ba53e527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:52.051905 21567 tablet_bootstrap.cc:492] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b: Bootstrap starting.
I20260808 02:12:52.053949 21568 tablet_bootstrap.cc:492] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126: Bootstrap starting.
I20260808 02:12:52.054369 21567 tablet_bootstrap.cc:654] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:52.055397 21567 log.cc:824] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:52.056243 21568 tablet_bootstrap.cc:654] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:52.057075 21568 log.cc:824] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:52.057469 21570 tablet_bootstrap.cc:492] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7: Bootstrap starting.
I20260808 02:12:52.058130 21567 tablet_bootstrap.cc:492] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b: No bootstrap required, opened a new log
I20260808 02:12:52.058363 21567 ts_tablet_manager.cc:1397] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260808 02:12:52.059165 21568 tablet_bootstrap.cc:492] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126: No bootstrap required, opened a new log
I20260808 02:12:52.059377 21568 ts_tablet_manager.cc:1397] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:12:52.059916 21570 tablet_bootstrap.cc:654] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:52.060788 21570 log.cc:824] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:52.062590 21567 raft_consensus.cc:348] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } }
I20260808 02:12:52.062881 21567 raft_consensus.cc:374] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:52.062944 21567 raft_consensus.cc:729] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ad5049927c4e1d80556710a9ff1b1b, State: Initialized, Role: FOLLOWER
I20260808 02:12:52.063179 21570 tablet_bootstrap.cc:492] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7: No bootstrap required, opened a new log
I20260808 02:12:52.063374 21570 ts_tablet_manager.cc:1397] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260808 02:12:52.063298 21568 raft_consensus.cc:348] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } }
I20260808 02:12:52.063532 21568 raft_consensus.cc:374] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:52.063601 21568 raft_consensus.cc:729] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fcf1968955e4146a69443b2105e3126, State: Initialized, Role: FOLLOWER
I20260808 02:12:52.063524 21567 consensus_queue.cc:260] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } }
I20260808 02:12:52.064008 21568 consensus_queue.cc:260] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } }
I20260808 02:12:52.064251 21247 heartbeater.cc:503] Master 127.18.184.190:42871 was elected leader, sending a full tablet report...
I20260808 02:12:52.064618 21567 ts_tablet_manager.cc:1428] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260808 02:12:52.064990 21568 ts_tablet_manager.cc:1428] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:12:52.065366 21396 heartbeater.cc:503] Master 127.18.184.190:42871 was elected leader, sending a full tablet report...
I20260808 02:12:52.067528 21570 raft_consensus.cc:348] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } }
I20260808 02:12:52.067808 21570 raft_consensus.cc:374] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:52.067878 21570 raft_consensus.cc:729] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17a3f03de0d4a68b2a8a3aa037398d7, State: Initialized, Role: FOLLOWER
I20260808 02:12:52.068301 21570 consensus_queue.cc:260] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } }
I20260808 02:12:52.069046 21544 heartbeater.cc:503] Master 127.18.184.190:42871 was elected leader, sending a full tablet report...
I20260808 02:12:52.069357 21570 ts_tablet_manager.cc:1428] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260808 02:12:52.127436 21575 raft_consensus.cc:482] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:52.127831 21575 raft_consensus.cc:504] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } }
I20260808 02:12:52.129218 21575 leader_election.cc:288] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fcf1968955e4146a69443b2105e3126 (127.18.184.130:40851), 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725)
I20260808 02:12:52.134248 21562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.129910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:36811 (local address 127.18.184.130:40851)
0808 02:12:52.130056 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.130061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.130203 (+ 142us) server_negotiation.cc:407] Connection header received
0808 02:12:52.130444 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.130451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.130514 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.130627 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:52.130970 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.131768 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.133060 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.133575 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.133731 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.133796 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.134012 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.134095 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:12:52.134275 21576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.129548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:40851 (local address 127.18.184.131:36811)
0808 02:12:52.130081 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.130096 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.130273 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.130640 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.130644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.130662 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:52.130812 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.130820 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.131894 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.131901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.132914 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.132925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.133731 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.133735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.133869 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.133891 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.133992 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.134072 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":1}
I20260808 02:12:52.134300 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.129918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:42715 (local address 127.18.184.129:44725)
0808 02:12:52.130096 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.130103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.130357 (+ 254us) server_negotiation.cc:407] Connection header received
0808 02:12:52.130613 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.130617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.130783 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.130896 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:52.131173 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.131834 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.133281 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.133721 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.133857 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.133909 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.134092 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.134168 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:12:52.134301 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.129548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:44725 (local address 127.18.184.131:42715)
0808 02:12:52.130246 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.130262 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.130409 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.130907 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.130910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.130927 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:52.131046 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.131052 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.131946 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.131951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.133107 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.133117 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.133849 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.133852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.133952 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.133975 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.134075 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.134168 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":79,"thread_start_us":153,"threads_started":1}
I20260808 02:12:52.135020 21198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f351e82c57949e384d4e119ba53e527" candidate_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51ad5049927c4e1d80556710a9ff1b1b" is_pre_election: true
I20260808 02:12:52.135074 21346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f351e82c57949e384d4e119ba53e527" candidate_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fcf1968955e4146a69443b2105e3126" is_pre_election: true
I20260808 02:12:52.135444 21198 raft_consensus.cc:2454] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17a3f03de0d4a68b2a8a3aa037398d7 in term 0.
I20260808 02:12:52.135458 21346 raft_consensus.cc:2454] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17a3f03de0d4a68b2a8a3aa037398d7 in term 0.
I20260808 02:12:52.136011 21425 leader_election.cc:302] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2fcf1968955e4146a69443b2105e3126, 51ad5049927c4e1d80556710a9ff1b1b, a17a3f03de0d4a68b2a8a3aa037398d7; no voters:
I20260808 02:12:52.136338 21575 raft_consensus.cc:2781] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:52.136426 21575 raft_consensus.cc:482] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:52.136483 21575 raft_consensus.cc:3037] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:52.137763 21575 raft_consensus.cc:504] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } }
I20260808 02:12:52.138206 21575 leader_election.cc:288] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [CANDIDATE]: Term 1 election: Requested vote from peers 2fcf1968955e4146a69443b2105e3126 (127.18.184.130:40851), 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725)
I20260808 02:12:52.138686 21346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f351e82c57949e384d4e119ba53e527" candidate_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fcf1968955e4146a69443b2105e3126"
I20260808 02:12:52.138684 21198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f351e82c57949e384d4e119ba53e527" candidate_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51ad5049927c4e1d80556710a9ff1b1b"
I20260808 02:12:52.138827 21346 raft_consensus.cc:3037] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:52.138865 21198 raft_consensus.cc:3037] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:52.140153 21198 raft_consensus.cc:2454] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a17a3f03de0d4a68b2a8a3aa037398d7 in term 1.
I20260808 02:12:52.140164 21346 raft_consensus.cc:2454] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a17a3f03de0d4a68b2a8a3aa037398d7 in term 1.
I20260808 02:12:52.140515 21426 leader_election.cc:302] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51ad5049927c4e1d80556710a9ff1b1b, a17a3f03de0d4a68b2a8a3aa037398d7; no voters:
I20260808 02:12:52.140735 21575 raft_consensus.cc:2781] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:52.140983 21575 raft_consensus.cc:686] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 LEADER]: Becoming Leader. State: Replica: a17a3f03de0d4a68b2a8a3aa037398d7, State: Running, Role: LEADER
I20260808 02:12:52.141382 21575 consensus_queue.cc:237] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } }
I20260808 02:12:52.145097 21029 catalog_manager.cc:5495] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131). New cstate: current_term: 1 leader_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } health_report { overall_health: UNKNOWN } } }
Aug 08 02:12:52 dist-test-slave-9gcw krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:52.189105 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.179084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:42871 (local address 127.0.0.1:42338)
0808 02:12:52.179442 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.179457 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.179567 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.179885 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.179887 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.180330 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:52.180439 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.180445 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.181704 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.181708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.182699 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.182710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.183189 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.183193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.183299 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.183924 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:52.183942 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:52.185812 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:52.187381 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:52.187390 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:52.187395 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:52.187638 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:52.187983 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:52.187985 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:52.187987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:52.188076 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:52.188503 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:52.188509 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:52.188618 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.188789 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.188930 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":128,"threads_started":1}
I20260808 02:12:52.189124 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.179182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.18.184.190:42871)
0808 02:12:52.179340 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.179346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.179546 (+ 200us) server_negotiation.cc:407] Connection header received
0808 02:12:52.179689 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.179693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.179751 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.179847 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:52.180579 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.181552 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.182832 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.183055 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.183145 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.185951 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:52.185954 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:52.185956 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:52.185978 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:52.187264 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.187786 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.187789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.187790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.187864 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.188168 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.188170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.188172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.188376 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:52.188456 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.188844 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.188995 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":70}
I20260808 02:12:52.193399 21029 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:42338:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:12:52.194213 21029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:52.200001 21325 tablet_service.cc:1467] Processing CreateTablet for tablet 14647658c0074929ad014cdd3bf25f06 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=17a7521ffb3f44669ba121b718775de5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:12:52.199903 21178 tablet_service.cc:1467] Processing CreateTablet for tablet 14647658c0074929ad014cdd3bf25f06 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=17a7521ffb3f44669ba121b718775de5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:12:52.200104 21468 tablet_service.cc:1467] Processing CreateTablet for tablet 14647658c0074929ad014cdd3bf25f06 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=17a7521ffb3f44669ba121b718775de5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:12:52.200798 21178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14647658c0074929ad014cdd3bf25f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:52.200798 21325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14647658c0074929ad014cdd3bf25f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:52.200814 21468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14647658c0074929ad014cdd3bf25f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:52.204068 21570 tablet_bootstrap.cc:492] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: Bootstrap starting.
I20260808 02:12:52.204339 21568 tablet_bootstrap.cc:492] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126: Bootstrap starting.
I20260808 02:12:52.204734 21567 tablet_bootstrap.cc:492] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b: Bootstrap starting.
I20260808 02:12:52.205415 21570 tablet_bootstrap.cc:654] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:52.205876 21568 tablet_bootstrap.cc:654] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:52.205876 21567 tablet_bootstrap.cc:654] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:52.207276 21567 tablet_bootstrap.cc:492] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b: No bootstrap required, opened a new log
I20260808 02:12:52.207394 21567 ts_tablet_manager.cc:1397] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:12:52.207574 21570 tablet_bootstrap.cc:492] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: No bootstrap required, opened a new log
I20260808 02:12:52.207588 21568 tablet_bootstrap.cc:492] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126: No bootstrap required, opened a new log
I20260808 02:12:52.207688 21570 ts_tablet_manager.cc:1397] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 02:12:52.207677 21568 ts_tablet_manager.cc:1397] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:12:52.207953 21567 raft_consensus.cc:348] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.208117 21567 raft_consensus.cc:374] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:52.208159 21567 raft_consensus.cc:729] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ad5049927c4e1d80556710a9ff1b1b, State: Initialized, Role: FOLLOWER
I20260808 02:12:52.208195 21568 raft_consensus.cc:348] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.208181 21570 raft_consensus.cc:348] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.208315 21570 raft_consensus.cc:374] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:52.208315 21568 raft_consensus.cc:374] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:52.208305 21567 consensus_queue.cc:260] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.208361 21568 raft_consensus.cc:729] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fcf1968955e4146a69443b2105e3126, State: Initialized, Role: FOLLOWER
I20260808 02:12:52.208361 21570 raft_consensus.cc:729] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17a3f03de0d4a68b2a8a3aa037398d7, State: Initialized, Role: FOLLOWER
I20260808 02:12:52.208504 21570 consensus_queue.cc:260] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.208597 21567 ts_tablet_manager.cc:1428] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:12:52.208618 21573 tablet_replica.cc:439] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } }
I20260808 02:12:52.208773 21573 tablet_replica.cc:442] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:52.208830 21570 ts_tablet_manager.cc:1428] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 02:12:52.209017 21568 consensus_queue.cc:260] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.209045 21575 tablet_replica.cc:439] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } }
I20260808 02:12:52.209223 21575 tablet_replica.cc:442] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:52.209401 21568 ts_tablet_manager.cc:1428] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:12:52.209383 21574 tablet_replica.cc:439] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } }
I20260808 02:12:52.209525 21574 tablet_replica.cc:442] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 02:12:52.220043 21545 tablet.cc:2369] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:12:52.222114 21397 tablet.cc:2369] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:12:52.229408 21248 tablet.cc:2369] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:12:52.438005 21575 raft_consensus.cc:482] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:52.438205 21575 raft_consensus.cc:504] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.438862 21575 leader_election.cc:288] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725), 2fcf1968955e4146a69443b2105e3126 (127.18.184.130:40851)
I20260808 02:12:52.439294 21198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14647658c0074929ad014cdd3bf25f06" candidate_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51ad5049927c4e1d80556710a9ff1b1b" is_pre_election: true
I20260808 02:12:52.439361 21346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14647658c0074929ad014cdd3bf25f06" candidate_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fcf1968955e4146a69443b2105e3126" is_pre_election: true
I20260808 02:12:52.439468 21198 raft_consensus.cc:2454] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17a3f03de0d4a68b2a8a3aa037398d7 in term 0.
I20260808 02:12:52.439577 21346 raft_consensus.cc:2454] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17a3f03de0d4a68b2a8a3aa037398d7 in term 0.
I20260808 02:12:52.439832 21426 leader_election.cc:302] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51ad5049927c4e1d80556710a9ff1b1b, a17a3f03de0d4a68b2a8a3aa037398d7; no voters:
I20260808 02:12:52.440042 21575 raft_consensus.cc:2781] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:52.440119 21575 raft_consensus.cc:482] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:52.440161 21575 raft_consensus.cc:3037] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:52.441275 21575 raft_consensus.cc:504] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.441659 21575 leader_election.cc:288] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [CANDIDATE]: Term 1 election: Requested vote from peers 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725), 2fcf1968955e4146a69443b2105e3126 (127.18.184.130:40851)
I20260808 02:12:52.442045 21198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14647658c0074929ad014cdd3bf25f06" candidate_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51ad5049927c4e1d80556710a9ff1b1b"
I20260808 02:12:52.442086 21346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14647658c0074929ad014cdd3bf25f06" candidate_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fcf1968955e4146a69443b2105e3126"
I20260808 02:12:52.442174 21198 raft_consensus.cc:3037] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:52.442250 21346 raft_consensus.cc:3037] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:52.443198 21198 raft_consensus.cc:2454] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a17a3f03de0d4a68b2a8a3aa037398d7 in term 1.
I20260808 02:12:52.443316 21346 raft_consensus.cc:2454] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a17a3f03de0d4a68b2a8a3aa037398d7 in term 1.
I20260808 02:12:52.443521 21426 leader_election.cc:302] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51ad5049927c4e1d80556710a9ff1b1b, a17a3f03de0d4a68b2a8a3aa037398d7; no voters:
I20260808 02:12:52.443717 21575 raft_consensus.cc:2781] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:52.443835 21575 raft_consensus.cc:686] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 LEADER]: Becoming Leader. State: Replica: a17a3f03de0d4a68b2a8a3aa037398d7, State: Running, Role: LEADER
I20260808 02:12:52.443984 21575 consensus_queue.cc:237] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.444803 21577 tablet_replica.cc:439] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: TxnStatusTablet state changed. Reason: New leader a17a3f03de0d4a68b2a8a3aa037398d7. Latest consensus state: current_term: 1 leader_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } }
I20260808 02:12:52.444921 21577 tablet_replica.cc:442] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:12:52.445290 21591 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:12:52.445545 21591 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:12:52.446009 21027 catalog_manager.cc:5495] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131). New cstate: current_term: 1 leader_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } health_report { overall_health: HEALTHY } } }
I20260808 02:12:52.507647 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.503469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:45053 (local address 127.0.0.1:58056)
0808 02:12:52.503740 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.503761 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.503878 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.504260 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.504263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.504271 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:52.504401 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.504407 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.505209 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.505213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.505934 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.505944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.506388 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.506391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.506502 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.506546 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:52.507338 (+ 792us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:52.507355 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.507426 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.507514 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260808 02:12:52.507681 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.503606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.18.184.131:45053)
0808 02:12:52.503832 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.503837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.503853 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:12:52.504069 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.504071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.504135 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.504244 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:52.504508 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.505062 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.506051 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.506272 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.506386 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.506708 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:52.507237 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:52.507364 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.507435 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.507497 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260808 02:12:52.513202 21346 raft_consensus.cc:1261] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Refusing update from remote peer a17a3f03de0d4a68b2a8a3aa037398d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:52.513253 21198 raft_consensus.cc:1261] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Refusing update from remote peer a17a3f03de0d4a68b2a8a3aa037398d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:52.513963 21577 consensus_queue.cc:1035] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:52.514190 21575 consensus_queue.cc:1035] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:52.518297 21573 tablet_replica.cc:439] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b: TxnStatusTablet state changed. Reason: New leader a17a3f03de0d4a68b2a8a3aa037398d7. Latest consensus state: current_term: 1 leader_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } }
I20260808 02:12:52.518424 21573 tablet_replica.cc:442] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:52.518842 21574 tablet_replica.cc:439] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126: TxnStatusTablet state changed. Reason: New leader a17a3f03de0d4a68b2a8a3aa037398d7. Latest consensus state: current_term: 1 leader_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } }
I20260808 02:12:52.518962 21574 tablet_replica.cc:442] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:52.521824 21577 tablet_replica.cc:439] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } }
I20260808 02:12:52.521945 21577 tablet_replica.cc:442] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:12:52.522410 21573 tablet_replica.cc:439] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } }
I20260808 02:12:52.522512 21573 tablet_replica.cc:442] T 14647658c0074929ad014cdd3bf25f06 P 51ad5049927c4e1d80556710a9ff1b1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:52.523187 21575 tablet_replica.cc:439] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } }
I20260808 02:12:52.523286 21575 tablet_replica.cc:442] T 14647658c0074929ad014cdd3bf25f06 P a17a3f03de0d4a68b2a8a3aa037398d7: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:12:52.523517 21594 mvcc.cc:205] Tried to move back new op lower bound from 7316091586607017984 to 7316091586332426240. Current Snapshot: MvccSnapshot[applied={T|T < 7316091586607017984}]
I20260808 02:12:52.523681 21596 mvcc.cc:205] Tried to move back new op lower bound from 7316091586607017984 to 7316091586332426240. Current Snapshot: MvccSnapshot[applied={T|T < 7316091586607017984}]
I20260808 02:12:52.524416 21574 tablet_replica.cc:439] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } } }
I20260808 02:12:52.524549 21574 tablet_replica.cc:442] T 14647658c0074929ad014cdd3bf25f06 P 2fcf1968955e4146a69443b2105e3126: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:52.526432 21595 mvcc.cc:205] Tried to move back new op lower bound from 7316091586607017984 to 7316091586332426240. Current Snapshot: MvccSnapshot[applied={T|T < 7316091586607017984}]
I20260808 02:12:52.527019 21577 consensus_queue.cc:1035] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:52.532224 21575 consensus_queue.cc:1035] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:52.541301 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.536564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58060 (local address 127.18.184.131:45053)
0808 02:12:52.536836 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.536842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.536962 (+ 120us) server_negotiation.cc:407] Connection header received
0808 02:12:52.537193 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.537199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.537279 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.537354 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:52.537703 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.538257 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.539749 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.540190 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.540322 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.540705 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:52.540798 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:52.540889 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.541037 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.541111 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260808 02:12:52.553433 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.549940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:45651 (local address 127.18.184.131:45053)
0808 02:12:52.550068 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.550072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.550301 (+ 229us) server_negotiation.cc:407] Connection header received
0808 02:12:52.550455 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.550458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.550503 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.550584 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:52.550836 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.551340 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.552271 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.552483 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.552593 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.552886 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:52.552954 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:52.553036 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.553210 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.553255 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":43}
I20260808 02:12:52.553445 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.549856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:45053 (local address 127.18.184.131:45651)
0808 02:12:52.550217 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.550230 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.550360 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.550591 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.550594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.550600 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:52.550729 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.550734 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.551451 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.551455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.552146 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.552157 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.552594 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.552597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.552686 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.552725 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:52.553037 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:52.553044 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.553187 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.553267 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":19,"thread_start_us":106,"threads_started":1}
I20260808 02:12:52.606570 21027 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:52.606840 21027 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:52.609688 21027 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6: Sending DeleteTablet for 3 replicas of tablet 0f351e82c57949e384d4e119ba53e527
I20260808 02:12:52.610510 21325 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f351e82c57949e384d4e119ba53e527 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:52 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:33394
I20260808 02:12:52.610598 21473 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f351e82c57949e384d4e119ba53e527 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:52 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:42044
I20260808 02:12:52.610836 21178 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f351e82c57949e384d4e119ba53e527 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:52 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:57584
I20260808 02:12:52.611282 21615 tablet_replica.cc:330] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7: stopping tablet replica
I20260808 02:12:52.611284 21614 tablet_replica.cc:330] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126: stopping tablet replica
I20260808 02:12:52.611622 21615 raft_consensus.cc:2229] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:12:52.611614 21614 raft_consensus.cc:2229] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:52.611833 21614 raft_consensus.cc:2258] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:52.611995 21615 raft_consensus.cc:2258] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:52.611987 21027 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:12:52.612421 21027 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:52.613332 21614 ts_tablet_manager.cc:1905] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:52.613332 21615 ts_tablet_manager.cc:1905] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:52.614944 21616 tablet_replica.cc:330] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b: stopping tablet replica
I20260808 02:12:52.615331 21616 raft_consensus.cc:2229] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:52.615628 21616 raft_consensus.cc:2258] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:52.616722 21615 ts_tablet_manager.cc:1918] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:52.616828 21615 log.cc:1196] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/0f351e82c57949e384d4e119ba53e527
I20260808 02:12:52.617231 21615 ts_tablet_manager.cc:1939] T 0f351e82c57949e384d4e119ba53e527 P a17a3f03de0d4a68b2a8a3aa037398d7: Deleting consensus metadata
I20260808 02:12:52.618559 21014 catalog_manager.cc:4841] TS a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131:45053): tablet 0f351e82c57949e384d4e119ba53e527 (table test-table [id=9c3fa31e946240ae99206215fe643f20]) successfully deleted
I20260808 02:12:52.619186 21325 tablet_service.cc:1467] Processing CreateTablet for tablet 1d4c1b1f50094ba49abaffb4d78071ab (DEFAULT_TABLE table=test-table [id=910b30c75d2d44b6b0e80f675c34633d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:52.619261 21473 tablet_service.cc:1467] Processing CreateTablet for tablet 1d4c1b1f50094ba49abaffb4d78071ab (DEFAULT_TABLE table=test-table [id=910b30c75d2d44b6b0e80f675c34633d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:52.619474 21325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d4c1b1f50094ba49abaffb4d78071ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:52.619546 21473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d4c1b1f50094ba49abaffb4d78071ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:52.620124 21614 ts_tablet_manager.cc:1918] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:52.620216 21614 log.cc:1196] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/0f351e82c57949e384d4e119ba53e527
I20260808 02:12:52.620535 21614 ts_tablet_manager.cc:1939] T 0f351e82c57949e384d4e119ba53e527 P 2fcf1968955e4146a69443b2105e3126: Deleting consensus metadata
I20260808 02:12:52.622043 21568 tablet_bootstrap.cc:492] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126: Bootstrap starting.
I20260808 02:12:52.622112 21570 tablet_bootstrap.cc:492] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7: Bootstrap starting.
I20260808 02:12:52.622892 21568 tablet_bootstrap.cc:654] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:52.623090 21570 tablet_bootstrap.cc:654] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:52.624222 21568 tablet_bootstrap.cc:492] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126: No bootstrap required, opened a new log
I20260808 02:12:52.624327 21568 ts_tablet_manager.cc:1397] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:12:52.624673 21178 tablet_service.cc:1467] Processing CreateTablet for tablet 1d4c1b1f50094ba49abaffb4d78071ab (DEFAULT_TABLE table=test-table [id=910b30c75d2d44b6b0e80f675c34633d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:52.624814 21568 raft_consensus.cc:348] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.624940 21568 raft_consensus.cc:374] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:52.624958 21178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d4c1b1f50094ba49abaffb4d78071ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:52.624965 21568 raft_consensus.cc:729] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fcf1968955e4146a69443b2105e3126, State: Initialized, Role: FOLLOWER
I20260808 02:12:52.625097 21568 consensus_queue.cc:260] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.625396 21568 ts_tablet_manager.cc:1428] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:52.627679 21567 tablet_bootstrap.cc:492] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b: Bootstrap starting.
I20260808 02:12:52.627950 21014 catalog_manager.cc:4841] TS 2fcf1968955e4146a69443b2105e3126 (127.18.184.130:40851): tablet 0f351e82c57949e384d4e119ba53e527 (table test-table [id=9c3fa31e946240ae99206215fe643f20]) successfully deleted
I20260808 02:12:52.628582 21567 tablet_bootstrap.cc:654] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:52.629436 21570 tablet_bootstrap.cc:492] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7: No bootstrap required, opened a new log
I20260808 02:12:52.629536 21570 ts_tablet_manager.cc:1397] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 02:12:52.630048 21570 raft_consensus.cc:348] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.630185 21570 raft_consensus.cc:374] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:52.630221 21570 raft_consensus.cc:729] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17a3f03de0d4a68b2a8a3aa037398d7, State: Initialized, Role: FOLLOWER
I20260808 02:12:52.630584 21567 tablet_bootstrap.cc:492] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b: No bootstrap required, opened a new log
I20260808 02:12:52.630676 21567 ts_tablet_manager.cc:1397] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:12:52.631115 21567 raft_consensus.cc:348] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.631273 21567 raft_consensus.cc:374] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:52.631317 21567 raft_consensus.cc:729] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ad5049927c4e1d80556710a9ff1b1b, State: Initialized, Role: FOLLOWER
I20260808 02:12:52.631247 21570 consensus_queue.cc:260] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.631439 21567 consensus_queue.cc:260] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.631603 21570 ts_tablet_manager.cc:1428] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260808 02:12:52.631788 21567 ts_tablet_manager.cc:1428] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:52.633847 21616 ts_tablet_manager.cc:1905] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:52.637982 21616 ts_tablet_manager.cc:1918] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:52.638100 21616 log.cc:1196] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/wal/wals/0f351e82c57949e384d4e119ba53e527
I20260808 02:12:52.638437 21616 ts_tablet_manager.cc:1939] T 0f351e82c57949e384d4e119ba53e527 P 51ad5049927c4e1d80556710a9ff1b1b: Deleting consensus metadata
I20260808 02:12:52.639901 21015 catalog_manager.cc:4841] TS 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725): tablet 0f351e82c57949e384d4e119ba53e527 (table test-table [id=9c3fa31e946240ae99206215fe643f20]) successfully deleted
I20260808 02:12:52.785710 21604 raft_consensus.cc:482] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:52.785944 21604 raft_consensus.cc:504] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.787302 21604 leader_election.cc:288] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725), a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131:45053)
I20260808 02:12:52.792088 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.787928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:60583 (local address 127.18.184.131:45053)
0808 02:12:52.788047 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.788052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.788176 (+ 124us) server_negotiation.cc:407] Connection header received
0808 02:12:52.788392 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.788395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.788449 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.788501 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:52.788889 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.789694 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.791069 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.791479 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.791628 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.791690 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.791887 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.791969 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:12:52.792105 21621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.787581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:45053 (local address 127.18.184.130:60583)
0808 02:12:52.788058 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.788072 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.788243 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.788653 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.788656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.788675 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:52.788766 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.788772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.789842 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.789845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.790941 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.790948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.791631 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.791634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.791742 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.791765 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.791845 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.791920 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":28,"thread_start_us":79,"threads_started":1}
I20260808 02:12:52.792111 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.787496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:44725 (local address 127.18.184.130:59363)
0808 02:12:52.787856 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.787872 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.787997 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.788362 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.788365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.788391 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:52.788506 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.788512 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.789461 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.789467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.790768 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.790779 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.791764 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.791767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.791869 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.791887 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.791950 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.792005 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260808 02:12:52.792454 21622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.787622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:59363 (local address 127.18.184.129:44725)
0808 02:12:52.787916 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.787933 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.787952 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:12:52.788166 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.788169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.788227 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.788317 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:52.788658 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.789344 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.790919 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.791548 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.791676 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.791728 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.792182 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.792288 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":61,"threads_started":1}
I20260808 02:12:52.792899 21198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d4c1b1f50094ba49abaffb4d78071ab" candidate_uuid: "2fcf1968955e4146a69443b2105e3126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51ad5049927c4e1d80556710a9ff1b1b" is_pre_election: true
I20260808 02:12:52.793084 21198 raft_consensus.cc:2454] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fcf1968955e4146a69443b2105e3126 in term 0.
I20260808 02:12:52.793221 21494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d4c1b1f50094ba49abaffb4d78071ab" candidate_uuid: "2fcf1968955e4146a69443b2105e3126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" is_pre_election: true
I20260808 02:12:52.793483 21494 raft_consensus.cc:2454] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fcf1968955e4146a69443b2105e3126 in term 0.
I20260808 02:12:52.793503 21277 leader_election.cc:302] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fcf1968955e4146a69443b2105e3126, 51ad5049927c4e1d80556710a9ff1b1b; no voters:
I20260808 02:12:52.793793 21604 raft_consensus.cc:2781] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:52.793885 21604 raft_consensus.cc:482] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:52.793927 21604 raft_consensus.cc:3037] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:52.795571 21604 raft_consensus.cc:504] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.796010 21604 leader_election.cc:288] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [CANDIDATE]: Term 1 election: Requested vote from peers 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725), a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131:45053)
I20260808 02:12:52.796428 21198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d4c1b1f50094ba49abaffb4d78071ab" candidate_uuid: "2fcf1968955e4146a69443b2105e3126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51ad5049927c4e1d80556710a9ff1b1b"
I20260808 02:12:52.796475 21494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d4c1b1f50094ba49abaffb4d78071ab" candidate_uuid: "2fcf1968955e4146a69443b2105e3126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7"
I20260808 02:12:52.796576 21198 raft_consensus.cc:3037] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:52.796643 21494 raft_consensus.cc:3037] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:52.797694 21198 raft_consensus.cc:2454] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fcf1968955e4146a69443b2105e3126 in term 1.
I20260808 02:12:52.797865 21494 raft_consensus.cc:2454] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fcf1968955e4146a69443b2105e3126 in term 1.
I20260808 02:12:52.798041 21277 leader_election.cc:302] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fcf1968955e4146a69443b2105e3126, 51ad5049927c4e1d80556710a9ff1b1b; no voters:
I20260808 02:12:52.798245 21604 raft_consensus.cc:2781] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:52.798388 21604 raft_consensus.cc:686] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 1 LEADER]: Becoming Leader. State: Replica: 2fcf1968955e4146a69443b2105e3126, State: Running, Role: LEADER
I20260808 02:12:52.798662 21604 consensus_queue.cc:237] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:52.801023 21027 catalog_manager.cc:5495] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fcf1968955e4146a69443b2105e3126 (127.18.184.130). New cstate: current_term: 1 leader_uuid: "2fcf1968955e4146a69443b2105e3126" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } health_report { overall_health: UNKNOWN } } }
I20260808 02:12:52.832858 21624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.828248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.18.184.130:40851)
0808 02:12:52.828592 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.828686 (+ 94us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.828703 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:12:52.828786 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.828788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.828848 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.829010 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:52.829357 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.830016 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.831007 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.831318 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.831459 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.831789 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:52.832294 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:52.832456 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.832546 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.832628 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":111,"threads_started":1}
I20260808 02:12:52.845328 21627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.841543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:45053 (local address 127.18.184.130:38247)
0808 02:12:52.841944 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.841957 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.842097 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.842393 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.842396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.842403 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:52.842544 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.842550 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.843351 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.843355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.844015 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.844025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.844517 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.844521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.844622 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.844672 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:52.844992 (+ 320us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:52.845002 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.845068 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.845174 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":140,"threads_started":1}
I20260808 02:12:52.845301 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.841703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:38247 (local address 127.18.184.131:45053)
0808 02:12:52.841884 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.841889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.842040 (+ 151us) server_negotiation.cc:407] Connection header received
0808 02:12:52.842234 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.842237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.842293 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.842380 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:52.842672 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.843229 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.844153 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.844390 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.844513 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.844827 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:52.844902 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:52.844985 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.845079 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.845129 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 02:12:52.851872 21494 raft_consensus.cc:1261] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Refusing update from remote peer 2fcf1968955e4146a69443b2105e3126: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:52.852403 21198 raft_consensus.cc:1261] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Refusing update from remote peer 2fcf1968955e4146a69443b2105e3126: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:52.852773 21604 consensus_queue.cc:1035] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:52.852994 21574 consensus_queue.cc:1035] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:52.871007 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.867518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:40851 (local address 127.18.184.131:50921)
0808 02:12:52.867698 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.867713 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.867805 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.868089 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.868094 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.868104 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:52.868216 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.868224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.869080 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.869082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.869741 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.869750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.870243 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.870247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.870326 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.870373 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:52.870677 (+ 304us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:52.870684 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.870754 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.870866 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 02:12:52.871008 21624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.867639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:50921 (local address 127.18.184.130:40851)
0808 02:12:52.867857 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.867862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.867877 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:12:52.867932 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.867935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.867981 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.868067 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:52.868429 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.868978 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.869901 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.870120 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.870255 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.870508 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:52.870579 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:52.870668 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.870794 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.870865 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
W20260808 02:12:52.871865 21595 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 02:12:54.434068 21027 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:54.434252 21027 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:54.436776 21027 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6: Sending DeleteTablet for 3 replicas of tablet 1d4c1b1f50094ba49abaffb4d78071ab
I20260808 02:12:54.437419 21178 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d4c1b1f50094ba49abaffb4d78071ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:54 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:57584
I20260808 02:12:54.437439 21325 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d4c1b1f50094ba49abaffb4d78071ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:54 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:33394
I20260808 02:12:54.437798 21655 tablet_replica.cc:330] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126: stopping tablet replica
I20260808 02:12:54.437928 21655 raft_consensus.cc:2229] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:12:54.438174 21655 raft_consensus.cc:2258] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:54.438741 21655 ts_tablet_manager.cc:1905] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:54.440626 21473 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d4c1b1f50094ba49abaffb4d78071ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:54 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:42044
I20260808 02:12:54.441009 21658 tablet_replica.cc:330] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7: stopping tablet replica
I20260808 02:12:54.441183 21658 raft_consensus.cc:2229] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:54.441335 21658 raft_consensus.cc:2258] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:54.441390 21655 ts_tablet_manager.cc:1918] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:12:54.441459 21655 log.cc:1196] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/1d4c1b1f50094ba49abaffb4d78071ab
I20260808 02:12:54.441838 21655 ts_tablet_manager.cc:1939] T 1d4c1b1f50094ba49abaffb4d78071ab P 2fcf1968955e4146a69443b2105e3126: Deleting consensus metadata
I20260808 02:12:54.441998 21656 tablet_replica.cc:330] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b: stopping tablet replica
I20260808 02:12:54.442096 21658 ts_tablet_manager.cc:1905] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:54.442245 21656 raft_consensus.cc:2229] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:54.442425 21656 raft_consensus.cc:2258] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:54.442564 21014 catalog_manager.cc:4841] TS 2fcf1968955e4146a69443b2105e3126 (127.18.184.130:40851): tablet 1d4c1b1f50094ba49abaffb4d78071ab (table test-table [id=910b30c75d2d44b6b0e80f675c34633d]) successfully deleted
I20260808 02:12:54.443193 21656 ts_tablet_manager.cc:1905] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:54.444855 21658 ts_tablet_manager.cc:1918] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:12:54.444957 21658 log.cc:1196] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/1d4c1b1f50094ba49abaffb4d78071ab
I20260808 02:12:54.445288 21658 ts_tablet_manager.cc:1939] T 1d4c1b1f50094ba49abaffb4d78071ab P a17a3f03de0d4a68b2a8a3aa037398d7: Deleting consensus metadata
I20260808 02:12:54.446051 21656 ts_tablet_manager.cc:1918] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:12:54.446154 21656 log.cc:1196] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/wal/wals/1d4c1b1f50094ba49abaffb4d78071ab
I20260808 02:12:54.446434 21014 catalog_manager.cc:4841] TS a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131:45053): tablet 1d4c1b1f50094ba49abaffb4d78071ab (table test-table [id=910b30c75d2d44b6b0e80f675c34633d]) successfully deleted
I20260808 02:12:54.446593 21656 ts_tablet_manager.cc:1939] T 1d4c1b1f50094ba49abaffb4d78071ab P 51ad5049927c4e1d80556710a9ff1b1b: Deleting consensus metadata
I20260808 02:12:54.447434 21015 catalog_manager.cc:4841] TS 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725): tablet 1d4c1b1f50094ba49abaffb4d78071ab (table test-table [id=910b30c75d2d44b6b0e80f675c34633d]) successfully deleted
Aug 08 02:12:54 dist-test-slave-9gcw krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155174, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:12:54 dist-test-slave-9gcw krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155174, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:54.473807 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.459345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47156 (local address 127.18.184.190:42871)
0808 02:12:54.459712 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.459722 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.459898 (+ 176us) server_negotiation.cc:407] Connection header received
0808 02:12:54.460086 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.460089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.460138 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.460267 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:54.461005 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.461650 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.462509 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.462757 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.462868 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.470247 (+ 7379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:54.470251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:54.470253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:54.470287 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:54.471844 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.472344 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.472346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.472348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.472431 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.472755 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.472758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.472759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.472971 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:54.473081 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.473473 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.473658 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":123,"threads_started":1}
I20260808 02:12:54.477066 21027 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:12:54.477650 21027 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:54.482699 21178 tablet_service.cc:1467] Processing CreateTablet for tablet 804dcd67ac6847dda2e97e78b8b2b66a (DEFAULT_TABLE table=test-table [id=4d735fc767f14776bd25fd65c2100d49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:54.482846 21325 tablet_service.cc:1467] Processing CreateTablet for tablet 804dcd67ac6847dda2e97e78b8b2b66a (DEFAULT_TABLE table=test-table [id=4d735fc767f14776bd25fd65c2100d49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:54.482810 21473 tablet_service.cc:1467] Processing CreateTablet for tablet 804dcd67ac6847dda2e97e78b8b2b66a (DEFAULT_TABLE table=test-table [id=4d735fc767f14776bd25fd65c2100d49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:54.483042 21178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 804dcd67ac6847dda2e97e78b8b2b66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:54.483117 21325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 804dcd67ac6847dda2e97e78b8b2b66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:54.483150 21473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 804dcd67ac6847dda2e97e78b8b2b66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:54.486024 21666 tablet_bootstrap.cc:492] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126: Bootstrap starting.
I20260808 02:12:54.486276 21667 tablet_bootstrap.cc:492] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b: Bootstrap starting.
I20260808 02:12:54.487157 21666 tablet_bootstrap.cc:654] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:54.487216 21667 tablet_bootstrap.cc:654] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:54.488423 21666 tablet_bootstrap.cc:492] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126: No bootstrap required, opened a new log
I20260808 02:12:54.488500 21666 ts_tablet_manager.cc:1397] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:12:54.488929 21666 raft_consensus.cc:348] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.489065 21666 raft_consensus.cc:374] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:54.489176 21666 raft_consensus.cc:729] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fcf1968955e4146a69443b2105e3126, State: Initialized, Role: FOLLOWER
I20260808 02:12:54.489336 21666 consensus_queue.cc:260] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.489688 21666 ts_tablet_manager.cc:1428] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:12:54.490466 21667 tablet_bootstrap.cc:492] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b: No bootstrap required, opened a new log
I20260808 02:12:54.490585 21667 ts_tablet_manager.cc:1397] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 02:12:54.491017 21667 raft_consensus.cc:348] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.491140 21667 raft_consensus.cc:374] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:54.491192 21667 raft_consensus.cc:729] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ad5049927c4e1d80556710a9ff1b1b, State: Initialized, Role: FOLLOWER
I20260808 02:12:54.491369 21667 consensus_queue.cc:260] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.491801 21667 ts_tablet_manager.cc:1428] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:54.494406 21668 tablet_bootstrap.cc:492] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7: Bootstrap starting.
I20260808 02:12:54.495329 21668 tablet_bootstrap.cc:654] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:54.496776 21668 tablet_bootstrap.cc:492] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7: No bootstrap required, opened a new log
I20260808 02:12:54.496889 21668 ts_tablet_manager.cc:1397] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:12:54.497373 21668 raft_consensus.cc:348] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.497511 21668 raft_consensus.cc:374] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:54.497561 21668 raft_consensus.cc:729] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17a3f03de0d4a68b2a8a3aa037398d7, State: Initialized, Role: FOLLOWER
I20260808 02:12:54.497704 21668 consensus_queue.cc:260] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.497982 21668 ts_tablet_manager.cc:1428] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:54.658157 21647 raft_consensus.cc:482] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:54.658366 21647 raft_consensus.cc:504] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.658942 21647 leader_election.cc:288] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725), a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131:45053)
I20260808 02:12:54.659502 21494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "804dcd67ac6847dda2e97e78b8b2b66a" candidate_uuid: "2fcf1968955e4146a69443b2105e3126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" is_pre_election: true
I20260808 02:12:54.659535 21197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "804dcd67ac6847dda2e97e78b8b2b66a" candidate_uuid: "2fcf1968955e4146a69443b2105e3126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51ad5049927c4e1d80556710a9ff1b1b" is_pre_election: true
I20260808 02:12:54.659698 21197 raft_consensus.cc:2454] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fcf1968955e4146a69443b2105e3126 in term 0.
I20260808 02:12:54.659783 21494 raft_consensus.cc:2454] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fcf1968955e4146a69443b2105e3126 in term 0.
I20260808 02:12:54.660140 21277 leader_election.cc:302] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2fcf1968955e4146a69443b2105e3126, 51ad5049927c4e1d80556710a9ff1b1b, a17a3f03de0d4a68b2a8a3aa037398d7; no voters:
I20260808 02:12:54.660423 21647 raft_consensus.cc:2781] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:54.660475 21647 raft_consensus.cc:482] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:54.660502 21647 raft_consensus.cc:3037] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:54.661623 21647 raft_consensus.cc:504] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.662004 21647 leader_election.cc:288] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [CANDIDATE]: Term 1 election: Requested vote from peers 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725), a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131:45053)
I20260808 02:12:54.662395 21197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "804dcd67ac6847dda2e97e78b8b2b66a" candidate_uuid: "2fcf1968955e4146a69443b2105e3126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51ad5049927c4e1d80556710a9ff1b1b"
I20260808 02:12:54.662416 21494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "804dcd67ac6847dda2e97e78b8b2b66a" candidate_uuid: "2fcf1968955e4146a69443b2105e3126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7"
I20260808 02:12:54.662536 21197 raft_consensus.cc:3037] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:54.662562 21494 raft_consensus.cc:3037] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:54.663488 21197 raft_consensus.cc:2454] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fcf1968955e4146a69443b2105e3126 in term 1.
I20260808 02:12:54.663571 21494 raft_consensus.cc:2454] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fcf1968955e4146a69443b2105e3126 in term 1.
I20260808 02:12:54.663832 21277 leader_election.cc:302] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fcf1968955e4146a69443b2105e3126, 51ad5049927c4e1d80556710a9ff1b1b; no voters:
I20260808 02:12:54.664001 21647 raft_consensus.cc:2781] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:54.664095 21647 raft_consensus.cc:686] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 1 LEADER]: Becoming Leader. State: Replica: 2fcf1968955e4146a69443b2105e3126, State: Running, Role: LEADER
I20260808 02:12:54.664281 21647 consensus_queue.cc:237] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.666008 21027 catalog_manager.cc:5495] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fcf1968955e4146a69443b2105e3126 (127.18.184.130). New cstate: current_term: 1 leader_uuid: "2fcf1968955e4146a69443b2105e3126" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } health_report { overall_health: UNKNOWN } } }
I20260808 02:12:54.691367 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.687937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47080 (local address 127.18.184.130:40851)
0808 02:12:54.688248 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.688256 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.688267 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:12:54.688304 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.688306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.688348 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.688439 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:54.688719 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.689326 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.690260 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.690496 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.690609 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.690880 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:54.690963 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:54.691044 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.691133 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.691195 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":87,"threads_started":1}
I20260808 02:12:54.697506 21197 raft_consensus.cc:1261] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Refusing update from remote peer 2fcf1968955e4146a69443b2105e3126: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:54.697517 21494 raft_consensus.cc:1261] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Refusing update from remote peer 2fcf1968955e4146a69443b2105e3126: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:54.697948 21574 consensus_queue.cc:1035] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:54.698154 21647 consensus_queue.cc:1035] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:54.740370 21029 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:54.740531 21029 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:54.743053 21029 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6: Sending DeleteTablet for 3 replicas of tablet 804dcd67ac6847dda2e97e78b8b2b66a
I20260808 02:12:54.743794 21178 tablet_service.cc:1514] Processing DeleteTablet for tablet 804dcd67ac6847dda2e97e78b8b2b66a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:54 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:57584
I20260808 02:12:54.743799 21325 tablet_service.cc:1514] Processing DeleteTablet for tablet 804dcd67ac6847dda2e97e78b8b2b66a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:54 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:33394
I20260808 02:12:54.743862 21473 tablet_service.cc:1514] Processing DeleteTablet for tablet 804dcd67ac6847dda2e97e78b8b2b66a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:54 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:42044
I20260808 02:12:54.743993 21655 tablet_replica.cc:330] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126: stopping tablet replica
I20260808 02:12:54.744050 21656 tablet_replica.cc:330] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b: stopping tablet replica
I20260808 02:12:54.744108 21655 raft_consensus.cc:2229] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:12:54.744223 21656 raft_consensus.cc:2229] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:54.744330 21655 raft_consensus.cc:2258] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:54.744382 21656 raft_consensus.cc:2258] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:54.744588 21658 tablet_replica.cc:330] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7: stopping tablet replica
I20260808 02:12:54.744575 21029 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 02:12:54.744732 21658 raft_consensus.cc:2229] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:54.744879 21658 raft_consensus.cc:2258] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 02:12:54.745047 21029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:54.745217 21655 ts_tablet_manager.cc:1905] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:54.745604 21656 ts_tablet_manager.cc:1905] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:54.745692 21658 ts_tablet_manager.cc:1905] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:54.747447 21655 ts_tablet_manager.cc:1918] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:54.747502 21655 log.cc:1196] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/804dcd67ac6847dda2e97e78b8b2b66a
I20260808 02:12:54.747706 21655 ts_tablet_manager.cc:1939] T 804dcd67ac6847dda2e97e78b8b2b66a P 2fcf1968955e4146a69443b2105e3126: Deleting consensus metadata
I20260808 02:12:54.748180 21656 ts_tablet_manager.cc:1918] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:54.748265 21656 log.cc:1196] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-0/wal/wals/804dcd67ac6847dda2e97e78b8b2b66a
I20260808 02:12:54.748406 21658 ts_tablet_manager.cc:1918] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:54.748476 21658 log.cc:1196] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/804dcd67ac6847dda2e97e78b8b2b66a
I20260808 02:12:54.748543 21656 ts_tablet_manager.cc:1939] T 804dcd67ac6847dda2e97e78b8b2b66a P 51ad5049927c4e1d80556710a9ff1b1b: Deleting consensus metadata
I20260808 02:12:54.748695 21014 catalog_manager.cc:4841] TS 2fcf1968955e4146a69443b2105e3126 (127.18.184.130:40851): tablet 804dcd67ac6847dda2e97e78b8b2b66a (table test-table [id=4d735fc767f14776bd25fd65c2100d49]) successfully deleted
I20260808 02:12:54.748766 21658 ts_tablet_manager.cc:1939] T 804dcd67ac6847dda2e97e78b8b2b66a P a17a3f03de0d4a68b2a8a3aa037398d7: Deleting consensus metadata
I20260808 02:12:54.749336 21015 catalog_manager.cc:4841] TS 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725): tablet 804dcd67ac6847dda2e97e78b8b2b66a (table test-table [id=4d735fc767f14776bd25fd65c2100d49]) successfully deleted
I20260808 02:12:54.749579 21014 catalog_manager.cc:4841] TS a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131:45053): tablet 804dcd67ac6847dda2e97e78b8b2b66a (table test-table [id=4d735fc767f14776bd25fd65c2100d49]) successfully deleted
I20260808 02:12:54.753404 21325 tablet_service.cc:1467] Processing CreateTablet for tablet 5636aba1c4de4344a69200e61391e07c (DEFAULT_TABLE table=test-table [id=ce14876ab49d4302bb12d3892c77c628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:54.753419 21178 tablet_service.cc:1467] Processing CreateTablet for tablet 5636aba1c4de4344a69200e61391e07c (DEFAULT_TABLE table=test-table [id=ce14876ab49d4302bb12d3892c77c628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:54.753746 21178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5636aba1c4de4344a69200e61391e07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:54.753756 21325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5636aba1c4de4344a69200e61391e07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:54.755689 21667 tablet_bootstrap.cc:492] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b: Bootstrap starting.
I20260808 02:12:54.755927 21666 tablet_bootstrap.cc:492] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126: Bootstrap starting.
I20260808 02:12:54.755924 21473 tablet_service.cc:1467] Processing CreateTablet for tablet 5636aba1c4de4344a69200e61391e07c (DEFAULT_TABLE table=test-table [id=ce14876ab49d4302bb12d3892c77c628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:54.756208 21473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5636aba1c4de4344a69200e61391e07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:54.756726 21667 tablet_bootstrap.cc:654] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:54.756872 21666 tablet_bootstrap.cc:654] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:54.758203 21667 tablet_bootstrap.cc:492] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b: No bootstrap required, opened a new log
I20260808 02:12:54.758284 21666 tablet_bootstrap.cc:492] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126: No bootstrap required, opened a new log
I20260808 02:12:54.758308 21667 ts_tablet_manager.cc:1397] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:12:54.758366 21666 ts_tablet_manager.cc:1397] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 02:12:54.758760 21668 tablet_bootstrap.cc:492] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7: Bootstrap starting.
I20260808 02:12:54.758793 21666 raft_consensus.cc:348] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.758793 21667 raft_consensus.cc:348] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.758924 21667 raft_consensus.cc:374] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:54.758934 21666 raft_consensus.cc:374] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:54.758973 21667 raft_consensus.cc:729] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ad5049927c4e1d80556710a9ff1b1b, State: Initialized, Role: FOLLOWER
I20260808 02:12:54.758973 21666 raft_consensus.cc:729] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fcf1968955e4146a69443b2105e3126, State: Initialized, Role: FOLLOWER
I20260808 02:12:54.759102 21666 consensus_queue.cc:260] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.759102 21667 consensus_queue.cc:260] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.759382 21667 ts_tablet_manager.cc:1428] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:54.759382 21666 ts_tablet_manager.cc:1428] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:12:54.759716 21668 tablet_bootstrap.cc:654] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:54.760824 21668 tablet_bootstrap.cc:492] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7: No bootstrap required, opened a new log
I20260808 02:12:54.760941 21668 ts_tablet_manager.cc:1397] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:12:54.761412 21668 raft_consensus.cc:348] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.761531 21668 raft_consensus.cc:374] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:54.761566 21668 raft_consensus.cc:729] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17a3f03de0d4a68b2a8a3aa037398d7, State: Initialized, Role: FOLLOWER
I20260808 02:12:54.761703 21668 consensus_queue.cc:260] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:54.761971 21668 ts_tablet_manager.cc:1428] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:55.108858 21647 raft_consensus.cc:482] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:55.109061 21647 raft_consensus.cc:504] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:55.109624 21647 leader_election.cc:288] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725), a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131:45053)
I20260808 02:12:55.110213 21494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5636aba1c4de4344a69200e61391e07c" candidate_uuid: "2fcf1968955e4146a69443b2105e3126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" is_pre_election: true
I20260808 02:12:55.110226 21198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5636aba1c4de4344a69200e61391e07c" candidate_uuid: "2fcf1968955e4146a69443b2105e3126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51ad5049927c4e1d80556710a9ff1b1b" is_pre_election: true
I20260808 02:12:55.110361 21494 raft_consensus.cc:2454] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fcf1968955e4146a69443b2105e3126 in term 0.
I20260808 02:12:55.110409 21198 raft_consensus.cc:2454] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fcf1968955e4146a69443b2105e3126 in term 0.
I20260808 02:12:55.110810 21277 leader_election.cc:302] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fcf1968955e4146a69443b2105e3126, 51ad5049927c4e1d80556710a9ff1b1b; no voters:
I20260808 02:12:55.111022 21647 raft_consensus.cc:2781] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:55.111085 21647 raft_consensus.cc:482] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:55.111140 21647 raft_consensus.cc:3037] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:55.112181 21647 raft_consensus.cc:504] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:55.112577 21647 leader_election.cc:288] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [CANDIDATE]: Term 1 election: Requested vote from peers 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725), a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131:45053)
I20260808 02:12:55.113083 21198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5636aba1c4de4344a69200e61391e07c" candidate_uuid: "2fcf1968955e4146a69443b2105e3126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51ad5049927c4e1d80556710a9ff1b1b"
I20260808 02:12:55.113081 21494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5636aba1c4de4344a69200e61391e07c" candidate_uuid: "2fcf1968955e4146a69443b2105e3126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7"
I20260808 02:12:55.113258 21198 raft_consensus.cc:3037] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:55.113258 21494 raft_consensus.cc:3037] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:55.114552 21494 raft_consensus.cc:2454] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fcf1968955e4146a69443b2105e3126 in term 1.
I20260808 02:12:55.114552 21198 raft_consensus.cc:2454] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fcf1968955e4146a69443b2105e3126 in term 1.
I20260808 02:12:55.114890 21277 leader_election.cc:302] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fcf1968955e4146a69443b2105e3126, 51ad5049927c4e1d80556710a9ff1b1b; no voters:
I20260808 02:12:55.115110 21647 raft_consensus.cc:2781] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:55.115250 21647 raft_consensus.cc:686] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 1 LEADER]: Becoming Leader. State: Replica: 2fcf1968955e4146a69443b2105e3126, State: Running, Role: LEADER
I20260808 02:12:55.115415 21647 consensus_queue.cc:237] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } }
I20260808 02:12:55.117322 21029 catalog_manager.cc:5495] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fcf1968955e4146a69443b2105e3126 (127.18.184.130). New cstate: current_term: 1 leader_uuid: "2fcf1968955e4146a69443b2105e3126" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fcf1968955e4146a69443b2105e3126" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 } health_report { overall_health: UNKNOWN } } }
I20260808 02:12:55.222911 21198 raft_consensus.cc:1261] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b [term 1 FOLLOWER]: Refusing update from remote peer 2fcf1968955e4146a69443b2105e3126: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:55.222949 21494 raft_consensus.cc:1261] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Refusing update from remote peer 2fcf1968955e4146a69443b2105e3126: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:55.223382 21574 consensus_queue.cc:1035] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a17a3f03de0d4a68b2a8a3aa037398d7" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 45053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:55.223578 21647 consensus_queue.cc:1035] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51ad5049927c4e1d80556710a9ff1b1b" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:12:55.236300 21638 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 02:12:56.803268 21029 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:56.803442 21029 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:56.806416 21029 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c31d9845d8884302b33caeca5515f0b6: Sending DeleteTablet for 3 replicas of tablet 5636aba1c4de4344a69200e61391e07c
I20260808 02:12:56.807077 21178 tablet_service.cc:1514] Processing DeleteTablet for tablet 5636aba1c4de4344a69200e61391e07c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:56 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:57584
I20260808 02:12:56.807056 21325 tablet_service.cc:1514] Processing DeleteTablet for tablet 5636aba1c4de4344a69200e61391e07c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:56 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:33394
I20260808 02:12:56.807304 21473 tablet_service.cc:1514] Processing DeleteTablet for tablet 5636aba1c4de4344a69200e61391e07c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:56 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:42044
I20260808 02:12:56.807525 21714 tablet_replica.cc:330] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126: stopping tablet replica
I20260808 02:12:56.807554 21715 tablet_replica.cc:330] T 5636aba1c4de4344a69200e61391e07c P 51ad5049927c4e1d80556710a9ff1b1b: stopping tablet replica
I20260808 02:12:56.807720 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 21109
I20260808 02:12:56.807744 21714 raft_consensus.cc:2229] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:12:56.812891 21714 raft_consensus.cc:2258] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:56.813719 21714 ts_tablet_manager.cc:1905] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:56.816200 21714 ts_tablet_manager.cc:1918] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:12:56.816282 21714 log.cc:1196] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/5636aba1c4de4344a69200e61391e07c
I20260808 02:12:56.816556 21716 tablet_replica.cc:330] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7: stopping tablet replica
I20260808 02:12:56.816602 21714 ts_tablet_manager.cc:1939] T 5636aba1c4de4344a69200e61391e07c P 2fcf1968955e4146a69443b2105e3126: Deleting consensus metadata
I20260808 02:12:56.816763 21716 raft_consensus.cc:2229] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:56.816938 21716 raft_consensus.cc:2258] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:56.817795 21716 ts_tablet_manager.cc:1905] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:56.817859 21014 catalog_manager.cc:4841] TS 2fcf1968955e4146a69443b2105e3126 (127.18.184.130:40851): tablet 5636aba1c4de4344a69200e61391e07c (table test-table [id=ce14876ab49d4302bb12d3892c77c628]) successfully deleted
W20260808 02:12:56.818989 21015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:44725 (error 108)
I20260808 02:12:56.819243 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 21257
I20260808 02:12:56.823944 21716 ts_tablet_manager.cc:1918] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:12:56.824035 21716 log.cc:1196] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/5636aba1c4de4344a69200e61391e07c
I20260808 02:12:56.824333 21716 ts_tablet_manager.cc:1939] T 5636aba1c4de4344a69200e61391e07c P a17a3f03de0d4a68b2a8a3aa037398d7: Deleting consensus metadata
I20260808 02:12:56.825448 21014 catalog_manager.cc:4841] TS a17a3f03de0d4a68b2a8a3aa037398d7 (127.18.184.131:45053): tablet 5636aba1c4de4344a69200e61391e07c (table test-table [id=ce14876ab49d4302bb12d3892c77c628]) successfully deleted
I20260808 02:12:56.825507 21718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.824925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:44725 (local address 127.0.0.1:33022)
0808 02:12:56.825287 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.825394 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":127,"threads_started":1}
W20260808 02:12:56.825758 21015 catalog_manager.cc:4584] TS 51ad5049927c4e1d80556710a9ff1b1b (127.18.184.129:44725): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5636aba1c4de4344a69200e61391e07c: Network error: Client connection negotiation failed: client connection to 127.18.184.129:44725: connect: Connection refused (error 111)
I20260808 02:12:56.830844 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 21405
I20260808 02:12:56.842242 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 20996
2026-08-08T02:12:56Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7195 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:12:56 dist-test-slave-9gcw krb5kdc[21721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:12:56 dist-test-slave-9gcw krb5kdc[21721](info): set up 2 sockets
Aug 08 02:12:56 dist-test-slave-9gcw krb5kdc[21721](info): commencing operation
krb5kdc: starting...
W20260808 02:12:57.994663 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:12:58 dist-test-slave-9gcw krb5kdc[21721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:12:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:12:58.133973 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:42263
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:42263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:58.249567 21736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:58.249855 21736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:58.249897 21736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:58.253849 21736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:12:58.253906 21736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:58.253925 21736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:58.253942 21736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:12:58.253957 21736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:12:58.259307 21736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:42263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:42263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:58.260761 21736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:58.261853 21736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:58.271251 21744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:58.271268 21742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:58.271471 21736 server_base.cc:1034] running on GCE node
W20260808 02:12:58.271268 21741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:58.272109 21736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:58.273406 21736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:58.274595 21736 hybrid_clock.cc:648] HybridClock initialized: now 1786155178274572 us; error 46 us; skew 500 ppm
Aug 08 02:12:58 dist-test-slave-9gcw krb5kdc[21721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:58.277544 21736 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:12:58.278546 21736 webserver.cc:460] Webserver started at http://127.18.184.190:41471/ using document root <none> and password file <none>
I20260808 02:12:58.279126 21736 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:58.279222 21736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:58.279457 21736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:58.281654 21736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "3f53e06289464a84875f1818c224d696"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-9gcw"
server_key: "15827771dc935938b55955663582da4c"
server_key_iv: "2c933506423c02e2d91cb7e9eaad8825"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.282239 21736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "3f53e06289464a84875f1818c224d696"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-9gcw"
server_key: "15827771dc935938b55955663582da4c"
server_key_iv: "2c933506423c02e2d91cb7e9eaad8825"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.286098 21736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:12:58.288527 21752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.289819 21736 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 02:12:58.289951 21736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "3f53e06289464a84875f1818c224d696"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-9gcw"
server_key: "15827771dc935938b55955663582da4c"
server_key_iv: "2c933506423c02e2d91cb7e9eaad8825"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.290061 21736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:58.310343 21736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:58.313959 21736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:58.314200 21736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:58.322166 21736 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:42263
I20260808 02:12:58.322178 21807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:42263 every 8 connection(s)
I20260808 02:12:58.323166 21736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:12:58.326154 21808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:58.331918 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 21736
I20260808 02:12:58.332046 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:12:58.332135 21808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696: Bootstrap starting.
I20260808 02:12:58.332340 19170 external_mini_cluster.cc:1428] Setting key 3fa85d5bf6b973129f737f4c1fa8f066
I20260808 02:12:58.334851 21808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:58.335711 21808 log.cc:824] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:58.338030 21808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696: No bootstrap required, opened a new log
Aug 08 02:12:58 dist-test-slave-9gcw krb5kdc[21721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:58.342684 21808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f53e06289464a84875f1818c224d696" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42263 } }
I20260808 02:12:58.342970 21808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:58.343029 21808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f53e06289464a84875f1818c224d696, State: Initialized, Role: FOLLOWER
I20260808 02:12:58.343449 21808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f53e06289464a84875f1818c224d696" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42263 } }
I20260808 02:12:58.343597 21808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:12:58.343672 21808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:12:58.343808 21808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:58.344691 21808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f53e06289464a84875f1818c224d696" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42263 } }
I20260808 02:12:58.345032 21808 leader_election.cc:302] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f53e06289464a84875f1818c224d696; no voters:
I20260808 02:12:58.345412 21808 leader_election.cc:288] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:12:58.345489 21813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:58.345703 21813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [term 1 LEADER]: Becoming Leader. State: Replica: 3f53e06289464a84875f1818c224d696, State: Running, Role: LEADER
I20260808 02:12:58.346011 21813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f53e06289464a84875f1818c224d696" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42263 } }
I20260808 02:12:58.346750 21808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:12:58.347213 21813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f53e06289464a84875f1818c224d696. Latest consensus state: current_term: 1 leader_uuid: "3f53e06289464a84875f1818c224d696" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f53e06289464a84875f1818c224d696" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42263 } } }
I20260808 02:12:58.347340 21813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:58.348510 21817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:12:58.348547 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.333702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59110 (local address 127.18.184.190:42263)
0808 02:12:58.334124 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:58.334148 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:58.334201 (+ 53us) server_negotiation.cc:407] Connection header received
0808 02:12:58.334862 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:58.334902 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:58.335288 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:58.335577 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:58.336506 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.337648 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.338788 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.339264 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.339412 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.342377 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:58.342383 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:58.342399 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:58.342431 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:58.344586 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.345188 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.345204 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.345211 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.345337 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.345677 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.345680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.345682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.346279 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:58.346489 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:58.346764 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 02:12:58.347093 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":158,"threads_started":1}
I20260808 02:12:58.348510 21815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f53e06289464a84875f1818c224d696" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f53e06289464a84875f1818c224d696" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42263 } } }
I20260808 02:12:58.349013 21815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:58.351862 21817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:12:58.357949 21817 catalog_manager.cc:1347] Generated new cluster ID: 6c9096944af14576bd2a99f5549c1458
I20260808 02:12:58.358037 21817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:12:58.370529 21817 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:12:58.371542 21817 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:12:58.377163 21817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f53e06289464a84875f1818c224d696: Generated new TSK 0
I20260808 02:12:58.377873 21817 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:12:58.417474 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:42263
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:58.547477 21836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:58.547860 21836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:58.547982 21836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:58.552824 21836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:58.552923 21836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:58.553030 21836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:12:58.558815 21836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:42263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:58.559968 21836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:58.561043 21836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:58.571977 21844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:58.572000 21836 server_base.cc:1034] running on GCE node
W20260808 02:12:58.571909 21841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:58.571918 21842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:58.572628 21836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:58.573377 21836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:58.574546 21836 hybrid_clock.cc:648] HybridClock initialized: now 1786155178574474 us; error 78 us; skew 500 ppm
Aug 08 02:12:58 dist-test-slave-9gcw krb5kdc[21721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:58.577471 21836 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:12:58.578347 21836 webserver.cc:460] Webserver started at http://127.18.184.129:34507/ using document root <none> and password file <none>
I20260808 02:12:58.578930 21836 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:58.578995 21836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:58.579210 21836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:58.581422 21836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "eabb235d06f043b390d1f4b93ee77e2f"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-9gcw"
server_key: "80b3351780320a2cf8b2a3a0d1d9c5a2"
server_key_iv: "71127a95045f1f54bf66f4bb7d9cf673"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.582015 21836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "eabb235d06f043b390d1f4b93ee77e2f"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-9gcw"
server_key: "80b3351780320a2cf8b2a3a0d1d9c5a2"
server_key_iv: "71127a95045f1f54bf66f4bb7d9cf673"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.586045 21836 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:12:58.588626 21852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.590080 21836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:12:58.590224 21836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "eabb235d06f043b390d1f4b93ee77e2f"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-9gcw"
server_key: "80b3351780320a2cf8b2a3a0d1d9c5a2"
server_key_iv: "71127a95045f1f54bf66f4bb7d9cf673"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.590346 21836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:58.607378 21836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:58.610507 21836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:58.610724 21836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:58.611359 21836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:58.612214 21836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:58.612293 21836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.612368 21836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:58.612414 21836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.621603 21836 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:45627
I20260808 02:12:58.621630 21973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:45627 every 8 connection(s)
I20260808 02:12:58.622535 21836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:12:58.625648 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 21836
I20260808 02:12:58.625769 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:12:58.626094 19170 external_mini_cluster.cc:1428] Setting key aa991f3daa182006d298898afbf3ef88
Aug 08 02:12:58 dist-test-slave-9gcw krb5kdc[21721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:12:58.638861 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.624486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:60671 (local address 127.18.184.190:42263)
0808 02:12:58.624735 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:58.624743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:58.625656 (+ 913us) server_negotiation.cc:407] Connection header received
0808 02:12:58.627281 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:58.627285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:58.627331 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:58.627503 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:58.629361 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.629949 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.631325 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.631589 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.631720 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.634938 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:58.634951 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:58.634953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:58.634981 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:58.636582 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.637168 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.637173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.637185 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.637275 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.637650 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.637653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.637655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.637861 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:58.637976 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:58.638518 (+ 542us) server_negotiation.cc:299] Negotiation successful
0808 02:12:58.638661 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:12:58.639916 21976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.624926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:42263 (local address 127.18.184.129:60671)
0808 02:12:58.625462 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:58.625526 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:58.627068 (+ 1542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:58.627739 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:58.627749 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:58.628309 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:58.629135 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:58.629198 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.630098 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.630102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:58.631168 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:58.631179 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.631769 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.631774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:58.632026 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.632721 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:58.632750 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:58.634735 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:58.636720 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:58.636727 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:58.636744 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:58.637029 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:58.637373 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:58.637378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:58.637380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:58.637537 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:58.637975 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:58.637982 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:58.638258 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 02:12:58.638457 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 02:12:58.638840 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":145,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
I20260808 02:12:58.641395 21974 heartbeater.cc:347] Connected to a master server at 127.18.184.190:42263
I20260808 02:12:58.641634 21974 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:58.642184 21974 heartbeater.cc:511] Master 127.18.184.190:42263 requested a full tablet report, sending...
I20260808 02:12:58.643781 21772 ts_manager.cc:194] Registered new tserver with Master: eabb235d06f043b390d1f4b93ee77e2f (127.18.184.129:45627)
I20260808 02:12:58.644953 21772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:60671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:12:58.660964 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:42263
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:58.778924 21979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:58.779188 21979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:58.779268 21979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:58.783406 21979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:58.783473 21979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:58.783565 21979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:12:58.789054 21979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:42263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:58.790210 21979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:58.791260 21979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:58.801436 21985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:58.801443 21986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:58.801443 21988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:58.801838 21979 server_base.cc:1034] running on GCE node
I20260808 02:12:58.802171 21979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:58.802722 21979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:58.803907 21979 hybrid_clock.cc:648] HybridClock initialized: now 1786155178803894 us; error 30 us; skew 500 ppm
Aug 08 02:12:58 dist-test-slave-9gcw krb5kdc[21721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:58.806789 21979 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:12:58.807578 21979 webserver.cc:460] Webserver started at http://127.18.184.130:34781/ using document root <none> and password file <none>
I20260808 02:12:58.808087 21979 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:58.808143 21979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:58.808310 21979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:58.810395 21979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "8fe883d510b74602a78c43b8022816c7"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-9gcw"
server_key: "d43553fc63607b7dfdbe1425932d5dbe"
server_key_iv: "a237d9a0a3642458d20842564fb6b5fb"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.810909 21979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "8fe883d510b74602a78c43b8022816c7"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-9gcw"
server_key: "d43553fc63607b7dfdbe1425932d5dbe"
server_key_iv: "a237d9a0a3642458d20842564fb6b5fb"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.814514 21979 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:12:58.816926 21995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.818239 21979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 02:12:58.818346 21979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "8fe883d510b74602a78c43b8022816c7"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-9gcw"
server_key: "d43553fc63607b7dfdbe1425932d5dbe"
server_key_iv: "a237d9a0a3642458d20842564fb6b5fb"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.818428 21979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:58.837741 21979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:58.840785 21979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:58.840953 21979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:58.841617 21979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:58.842442 21979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:58.842490 21979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.842536 21979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:58.842558 21979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.852770 21979 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:33005
I20260808 02:12:58.852797 22115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:33005 every 8 connection(s)
I20260808 02:12:58.853804 21979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:12:58.858721 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 21979
I20260808 02:12:58.858847 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:12:58.859135 19170 external_mini_cluster.cc:1428] Setting key fe1f79d6494a5157d7943e0fb9077794
Aug 08 02:12:58 dist-test-slave-9gcw krb5kdc[21721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:58.868670 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.855930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:39289 (local address 127.18.184.190:42263)
0808 02:12:58.856066 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:58.856075 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:58.856976 (+ 901us) server_negotiation.cc:407] Connection header received
0808 02:12:58.858434 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:58.858438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:58.858504 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:58.858594 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:58.860109 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.860714 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.861977 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.862199 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.862312 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.864862 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:58.864869 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:58.864875 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:58.864913 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:58.866535 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.867081 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.867085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.867087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.867176 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.867537 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.867541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.867543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.867736 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:58.867846 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:58.868370 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 02:12:58.868501 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:12:58.869630 22118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.856297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:42263 (local address 127.18.184.130:39289)
0808 02:12:58.856804 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:58.856851 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:58.858190 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:58.858813 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:58.858824 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:58.859346 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:58.859964 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:58.859983 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.860863 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.860867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:58.861839 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:58.861849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.862365 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.862369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:58.862539 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.863162 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:58.863180 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:58.864673 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:58.866664 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:58.866671 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:58.866687 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:58.866971 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:58.867272 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:58.867276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:58.867278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:58.867440 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:58.867847 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:58.867857 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:58.868119 (+ 262us) client_negotiation.cc:769] Sending connection context
0808 02:12:58.868322 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 02:12:58.868675 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":143,"threads_started":1}
I20260808 02:12:58.871232 22116 heartbeater.cc:347] Connected to a master server at 127.18.184.190:42263
I20260808 02:12:58.871523 22116 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
I20260808 02:12:58.872166 22116 heartbeater.cc:511] Master 127.18.184.190:42263 requested a full tablet report, sending...
I20260808 02:12:58.873644 21772 ts_manager.cc:194] Registered new tserver with Master: 8fe883d510b74602a78c43b8022816c7 (127.18.184.130:33005)
I20260808 02:12:58.874586 21772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:39289
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:12:58.891867 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:42263
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:59.020073 22121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:59.020354 22121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:59.020401 22121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:59.024610 22121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:59.024688 22121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:59.024749 22121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:12:59.030280 22121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:42263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:12:59.031459 22121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:59.032557 22121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:59.043129 22129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:59.043134 22127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:59.043128 22126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:59.043395 22121 server_base.cc:1034] running on GCE node
I20260808 02:12:59.044066 22121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:59.044780 22121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:59.045994 22121 hybrid_clock.cc:648] HybridClock initialized: now 1786155179045933 us; error 74 us; skew 500 ppm
Aug 08 02:12:59 dist-test-slave-9gcw krb5kdc[21721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155179, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:59.049654 22121 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:12:59.050776 22121 webserver.cc:460] Webserver started at http://127.18.184.131:37219/ using document root <none> and password file <none>
I20260808 02:12:59.051365 22121 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:59.051427 22121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:59.051612 22121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:59.053925 22121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "9a780324ba89458583539df174e99e6c"
format_stamp: "Formatted at 2026-08-08 02:12:59 on dist-test-slave-9gcw"
server_key: "57680b25c8ff3141d17799f42156c47a"
server_key_iv: "71e4ddf738be747b31b1b3bd64ab6e2d"
server_key_version: "encryptionkey@0"
I20260808 02:12:59.054474 22121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "9a780324ba89458583539df174e99e6c"
format_stamp: "Formatted at 2026-08-08 02:12:59 on dist-test-slave-9gcw"
server_key: "57680b25c8ff3141d17799f42156c47a"
server_key_iv: "71e4ddf738be747b31b1b3bd64ab6e2d"
server_key_version: "encryptionkey@0"
I20260808 02:12:59.058429 22121 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:12:59.061300 22137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:59.062644 22121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:12:59.062834 22121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "9a780324ba89458583539df174e99e6c"
format_stamp: "Formatted at 2026-08-08 02:12:59 on dist-test-slave-9gcw"
server_key: "57680b25c8ff3141d17799f42156c47a"
server_key_iv: "71e4ddf738be747b31b1b3bd64ab6e2d"
server_key_version: "encryptionkey@0"
I20260808 02:12:59.062959 22121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:59.086082 22121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:59.089484 22121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:59.089692 22121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:59.090358 22121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:59.091215 22121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:59.091284 22121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:59.091356 22121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:59.091396 22121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:59.101053 22121 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:43391
I20260808 02:12:59.101115 22257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:43391 every 8 connection(s)
I20260808 02:12:59.102052 22121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:12:59.110531 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 22121
I20260808 02:12:59.110698 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:12:59.111081 19170 external_mini_cluster.cc:1428] Setting key 7d42210fe2d51b6bfb5db3de0b7cee50
Aug 08 02:12:59 dist-test-slave-9gcw krb5kdc[21721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155179, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:12:59.122830 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.109368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:53045 (local address 127.18.184.190:42263)
0808 02:12:59.109593 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.109601 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.109625 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:12:59.109697 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.109701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.109768 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.109878 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:59.111315 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.112070 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.114272 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.114484 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.114563 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.118457 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:59.118464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:59.118468 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:59.118507 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:59.120418 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:59.121012 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:59.121022 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:59.121023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:59.121107 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:59.121557 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:59.121566 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:59.121568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:59.121781 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:59.121909 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.122484 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.122623 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:12:59.124344 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.104515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:42263 (local address 127.18.184.131:53045)
0808 02:12:59.105201 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.105286 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.107097 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.110124 (+ 3027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.110132 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.110535 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:59.111142 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.111164 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.112231 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.112237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.114123 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.114139 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.114670 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.114677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.114948 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.115699 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:59.115729 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:59.118195 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:59.120563 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:59.120570 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:59.120587 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:59.120887 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:59.121264 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:59.121267 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:59.121270 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:59.121437 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:59.121910 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:59.121919 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:59.122214 (+ 295us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.122421 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.122796 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":171,"threads_started":1}
I20260808 02:12:59.126142 22258 heartbeater.cc:347] Connected to a master server at 127.18.184.190:42263
I20260808 02:12:59.126533 22258 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:59.127298 22258 heartbeater.cc:511] Master 127.18.184.190:42263 requested a full tablet report, sending...
I20260808 02:12:59.128444 21772 ts_manager.cc:194] Registered new tserver with Master: 9a780324ba89458583539df174e99e6c (127.18.184.131:43391)
I20260808 02:12:59.129040 21772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:53045
I20260808 02:12:59.137858 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:12:59.159020 21811 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:12:59.157384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59112 (local address 127.18.184.190:42263)
0808 02:12:59.157574 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.157581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.157699 (+ 118us) server_negotiation.cc:407] Connection header received
0808 02:12:59.157823 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.157829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.157897 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.158013 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:59.158826 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59112: BlockingRecv error: recv got EOF from 127.0.0.1:59112 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:12:59.159925 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 21836
I20260808 02:12:59.174242 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 21979
I20260808 02:12:59.182737 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 22121
I20260808 02:12:59.189419 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 21736
2026-08-08T02:12:59Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2343 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:12:59 dist-test-slave-9gcw krb5kdc[22271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:12:59 dist-test-slave-9gcw krb5kdc[22271](info): set up 2 sockets
Aug 08 02:12:59 dist-test-slave-9gcw krb5kdc[22271](info): commencing operation
krb5kdc: starting...
W20260808 02:13:00.291234 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:13:00 dist-test-slave-9gcw krb5kdc[22271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:13:00.432600 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:37407
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:36845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:37407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:00.553432 22284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:00.553742 22284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:00.553784 22284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:00.558182 22284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:00.558272 22284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:00.558293 22284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:00.558311 22284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:00.558326 22284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:00.563916 22284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:36845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:37407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:37407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:00.565271 22284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:00.566380 22284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:00.575742 22294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:00.575834 22290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:00.575886 22291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:00.575909 22284 server_base.cc:1034] running on GCE node
I20260808 02:13:00.576682 22284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:00.577975 22284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:00.579193 22284 hybrid_clock.cc:648] HybridClock initialized: now 1786155180579170 us; error 41 us; skew 500 ppm
Aug 08 02:13:00 dist-test-slave-9gcw krb5kdc[22271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155180, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:00.587917 22284 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:13:00.589087 22284 webserver.cc:460] Webserver started at http://127.18.184.190:44449/ using document root <none> and password file <none>
I20260808 02:13:00.589754 22284 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:00.589856 22284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:00.590175 22284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:00.592528 22284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "29cd13d6791c430a8ee8eb3f24785a0e"
format_stamp: "Formatted at 2026-08-08 02:13:00 on dist-test-slave-9gcw"
server_key: "fd8d66acc2e4354401a0529c475e2c77"
server_key_iv: "5da25090831fb6fba1e4242da09d9723"
server_key_version: "encryptionkey@0"
I20260808 02:13:00.593207 22284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "29cd13d6791c430a8ee8eb3f24785a0e"
format_stamp: "Formatted at 2026-08-08 02:13:00 on dist-test-slave-9gcw"
server_key: "fd8d66acc2e4354401a0529c475e2c77"
server_key_iv: "5da25090831fb6fba1e4242da09d9723"
server_key_version: "encryptionkey@0"
I20260808 02:13:00.597355 22284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:13:00.599903 22302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:00.601329 22284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:00.601468 22284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "29cd13d6791c430a8ee8eb3f24785a0e"
format_stamp: "Formatted at 2026-08-08 02:13:00 on dist-test-slave-9gcw"
server_key: "fd8d66acc2e4354401a0529c475e2c77"
server_key_iv: "5da25090831fb6fba1e4242da09d9723"
server_key_version: "encryptionkey@0"
I20260808 02:13:00.601586 22284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:00.618822 22284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:00.622588 22284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:00.622859 22284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:00.630534 22284 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:37407
I20260808 02:13:00.630857 22360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:37407 every 8 connection(s)
I20260808 02:13:00.631599 22284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:00.634716 22361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:00.640900 22361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e: Bootstrap starting.
I20260808 02:13:00.641258 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 22284
I20260808 02:13:00.641405 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:00.641705 19170 external_mini_cluster.cc:1428] Setting key d7a74c86e8ce1f6e2b8a78b66d74065d
I20260808 02:13:00.643855 22361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:00.644630 22361 log.cc:824] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:00.646864 22361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e: No bootstrap required, opened a new log
I20260808 02:13:00.651477 22361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29cd13d6791c430a8ee8eb3f24785a0e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37407 } }
I20260808 02:13:00.651734 22361 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 08 02:13:00 dist-test-slave-9gcw krb5kdc[22271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:00.651811 22361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29cd13d6791c430a8ee8eb3f24785a0e, State: Initialized, Role: FOLLOWER
I20260808 02:13:00.652243 22361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29cd13d6791c430a8ee8eb3f24785a0e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37407 } }
I20260808 02:13:00.652390 22361 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:00.652452 22361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:00.652558 22361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:00.653784 22361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29cd13d6791c430a8ee8eb3f24785a0e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37407 } }
I20260808 02:13:00.654116 22361 leader_election.cc:302] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29cd13d6791c430a8ee8eb3f24785a0e; no voters:
I20260808 02:13:00.654508 22361 leader_election.cc:288] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:00.654588 22366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:00.654780 22366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [term 1 LEADER]: Becoming Leader. State: Replica: 29cd13d6791c430a8ee8eb3f24785a0e, State: Running, Role: LEADER
I20260808 02:13:00.655076 22366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29cd13d6791c430a8ee8eb3f24785a0e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37407 } }
I20260808 02:13:00.655866 22361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:00.656862 22368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29cd13d6791c430a8ee8eb3f24785a0e. Latest consensus state: current_term: 1 leader_uuid: "29cd13d6791c430a8ee8eb3f24785a0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29cd13d6791c430a8ee8eb3f24785a0e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37407 } } }
I20260808 02:13:00.656999 22368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:00.657495 22367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29cd13d6791c430a8ee8eb3f24785a0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29cd13d6791c430a8ee8eb3f24785a0e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37407 } } }
I20260808 02:13:00.657608 22367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:00.658910 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:00.643156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36140 (local address 127.18.184.190:37407)
0808 02:13:00.643595 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:00.643613 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:00.643651 (+ 38us) server_negotiation.cc:407] Connection header received
0808 02:13:00.644403 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:00.644423 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:00.644806 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:00.645222 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:00.646147 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:00.647341 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:00.648541 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:00.649067 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:00.649283 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:00.652518 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:00.652526 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:00.652545 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:00.652582 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:00.654751 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:00.655375 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:00.655381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:00.655388 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:00.655526 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:00.655863 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:00.655868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:00.655871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:00.656476 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:00.656768 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:00.657087 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 02:13:00.657477 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":136,"threads_started":1}
W20260808 02:13:00.660991 22381 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:13:00.661080 22381 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:13:00.661192 22375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:00.662123 22375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:00.662294 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:13:00.667668 22375 catalog_manager.cc:1347] Generated new cluster ID: 5b8e2815ba1d41cdb3f96167f43383a7
I20260808 02:13:00.667742 22375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:00.696488 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:00.689869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36146 (local address 127.18.184.190:37407)
0808 02:13:00.690019 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:00.690024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:00.690242 (+ 218us) server_negotiation.cc:407] Connection header received
0808 02:13:00.690387 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:00.690390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:00.690450 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:00.690542 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:00.691171 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:00.692530 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:00.693735 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:00.694154 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:00.694262 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:00.695240 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:00.695244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:00.695247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:00.695961 (+ 714us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:00.696000 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:00.696022 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:00.696124 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:00.696227 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 02:13:00.696311 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:13:00.699982 22375 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:00.700947 22375 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:00.707571 22375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29cd13d6791c430a8ee8eb3f24785a0e: Generated new TSK 0
I20260808 02:13:00.708211 22375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:00.715485 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 22284
2026-08-08T02:13:00Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1524 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:00 dist-test-slave-9gcw krb5kdc[22395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:00 dist-test-slave-9gcw krb5kdc[22395](info): set up 2 sockets
Aug 08 02:13:00 dist-test-slave-9gcw krb5kdc[22395](info): commencing operation
krb5kdc: starting...
W20260808 02:13:01.813850 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:13:01 dist-test-slave-9gcw krb5kdc[22395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:13:01.952814 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:38515
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:38515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:02.078497 22408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:02.078817 22408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:02.078861 22408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:02.083130 22408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:02.083206 22408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:02.083227 22408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:02.083246 22408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:02.083262 22408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:02.088976 22408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:38515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:38515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:02.090215 22408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:02.091296 22408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:02.100551 22416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:02.100543 22414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:02.100543 22413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:02.101305 22408 server_base.cc:1034] running on GCE node
I20260808 02:13:02.101943 22408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:02.103201 22408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:02.104408 22408 hybrid_clock.cc:648] HybridClock initialized: now 1786155182104383 us; error 50 us; skew 500 ppm
Aug 08 02:13:02 dist-test-slave-9gcw krb5kdc[22395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:02.107600 22408 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:13:02.108662 22408 webserver.cc:460] Webserver started at http://127.18.184.190:39959/ using document root <none> and password file <none>
I20260808 02:13:02.109361 22408 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:02.109462 22408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:02.109727 22408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:02.112044 22408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "9710b912b5e74d428a031e2d5755e2f7"
format_stamp: "Formatted at 2026-08-08 02:13:02 on dist-test-slave-9gcw"
server_key: "1e5de10a21c29a0a301c989d8bcecbb2"
server_key_iv: "7376480de8bd88590ba6d9883d3ac3c6"
server_key_version: "encryptionkey@0"
I20260808 02:13:02.112673 22408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "9710b912b5e74d428a031e2d5755e2f7"
format_stamp: "Formatted at 2026-08-08 02:13:02 on dist-test-slave-9gcw"
server_key: "1e5de10a21c29a0a301c989d8bcecbb2"
server_key_iv: "7376480de8bd88590ba6d9883d3ac3c6"
server_key_version: "encryptionkey@0"
I20260808 02:13:02.117013 22408 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 02:13:02.119876 22424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:02.121543 22408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:02.121709 22408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "9710b912b5e74d428a031e2d5755e2f7"
format_stamp: "Formatted at 2026-08-08 02:13:02 on dist-test-slave-9gcw"
server_key: "1e5de10a21c29a0a301c989d8bcecbb2"
server_key_iv: "7376480de8bd88590ba6d9883d3ac3c6"
server_key_version: "encryptionkey@0"
I20260808 02:13:02.121840 22408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:02.141876 22408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:02.145478 22408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:02.145735 22408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:02.153395 22408 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:38515
I20260808 02:13:02.153736 22481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:38515 every 8 connection(s)
I20260808 02:13:02.154536 22408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:02.157891 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:02.161445 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 22408
I20260808 02:13:02.161571 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:02.161865 19170 external_mini_cluster.cc:1428] Setting key 3477cb200be8b0201a36b2b7a1e4e198
I20260808 02:13:02.165565 22482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7: Bootstrap starting.
I20260808 02:13:02.168534 22482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:02.169570 22482 log.cc:824] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:02.171948 22482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7: No bootstrap required, opened a new log
Aug 08 02:13:02 dist-test-slave-9gcw krb5kdc[22395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:02.177205 22482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9710b912b5e74d428a031e2d5755e2f7" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38515 } }
I20260808 02:13:02.177528 22482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:02.177589 22482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9710b912b5e74d428a031e2d5755e2f7, State: Initialized, Role: FOLLOWER
I20260808 02:13:02.178095 22482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9710b912b5e74d428a031e2d5755e2f7" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38515 } }
I20260808 02:13:02.178265 22482 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:02.178344 22482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:02.178444 22482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:02.178596 22485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:02.163413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39998 (local address 127.18.184.190:38515)
0808 02:13:02.163894 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:02.163921 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:02.163971 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:13:02.164845 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:02.164871 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:02.165362 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:02.165767 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:02.166533 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.168119 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.169124 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.169863 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.170052 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:02.172741 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:02.172747 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:02.172765 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:02.172800 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:02.175037 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:02.175520 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:02.175526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:02.175533 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:02.175699 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:02.176037 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:02.176040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:02.176042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:02.176646 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:02.176865 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:02.177193 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 02:13:02.177515 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":113,"threads_started":1}
I20260808 02:13:02.179538 22482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9710b912b5e74d428a031e2d5755e2f7" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38515 } }
I20260808 02:13:02.179901 22482 leader_election.cc:302] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9710b912b5e74d428a031e2d5755e2f7; no voters:
I20260808 02:13:02.180299 22482 leader_election.cc:288] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:02.180351 22487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:02.180553 22487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [term 1 LEADER]: Becoming Leader. State: Replica: 9710b912b5e74d428a031e2d5755e2f7, State: Running, Role: LEADER
I20260808 02:13:02.180878 22487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9710b912b5e74d428a031e2d5755e2f7" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38515 } }
I20260808 02:13:02.181659 22482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:02.182196 22489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9710b912b5e74d428a031e2d5755e2f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9710b912b5e74d428a031e2d5755e2f7" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38515 } } }
I20260808 02:13:02.182387 22489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:02.182754 22488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9710b912b5e74d428a031e2d5755e2f7. Latest consensus state: current_term: 1 leader_uuid: "9710b912b5e74d428a031e2d5755e2f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9710b912b5e74d428a031e2d5755e2f7" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38515 } } }
I20260808 02:13:02.182880 22488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7 [sys.catalog]: This master's current role is: LEADER
W20260808 02:13:02.186911 22502 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:13:02.186987 22502 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:13:02.187089 22492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:02.187777 22492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:02.193914 22492 catalog_manager.cc:1347] Generated new cluster ID: 70c65143453d443389b6e2db26137b52
I20260808 02:13:02.193991 22492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:02.209733 22492 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:02.210662 22492 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:02.224414 22492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9710b912b5e74d428a031e2d5755e2f7: Generated new TSK 0
I20260808 02:13:02.225198 22492 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:13:02.256676 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:38515
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:02.373322 22508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:02.373600 22508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:02.373644 22508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:02.377657 22508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:02.377722 22508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:02.377784 22508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:02.383463 22508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:38515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:02.384690 22508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:02.385789 22508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:02.396121 22516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:02.396236 22508 server_base.cc:1034] running on GCE node
W20260808 02:13:02.396103 22513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:02.396103 22514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:02.396816 22508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:02.397652 22508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:02.398859 22508 hybrid_clock.cc:648] HybridClock initialized: now 1786155182398815 us; error 65 us; skew 500 ppm
Aug 08 02:13:02 dist-test-slave-9gcw krb5kdc[22395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:02.401937 22508 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:13:02.402931 22508 webserver.cc:460] Webserver started at http://127.18.184.129:38087/ using document root <none> and password file <none>
I20260808 02:13:02.403538 22508 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:02.403604 22508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:02.403831 22508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:02.406157 22508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "9ab3451d830b463d985f8b20e8f00a42"
format_stamp: "Formatted at 2026-08-08 02:13:02 on dist-test-slave-9gcw"
server_key: "30e288e7113a7ed2bb5339c988e42f10"
server_key_iv: "97d9b391397687f01d1c8225040b0687"
server_key_version: "encryptionkey@0"
I20260808 02:13:02.406770 22508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "9ab3451d830b463d985f8b20e8f00a42"
format_stamp: "Formatted at 2026-08-08 02:13:02 on dist-test-slave-9gcw"
server_key: "30e288e7113a7ed2bb5339c988e42f10"
server_key_iv: "97d9b391397687f01d1c8225040b0687"
server_key_version: "encryptionkey@0"
I20260808 02:13:02.410678 22508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:13:02.413455 22525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:02.414804 22508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:02.414935 22508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "9ab3451d830b463d985f8b20e8f00a42"
format_stamp: "Formatted at 2026-08-08 02:13:02 on dist-test-slave-9gcw"
server_key: "30e288e7113a7ed2bb5339c988e42f10"
server_key_iv: "97d9b391397687f01d1c8225040b0687"
server_key_version: "encryptionkey@0"
I20260808 02:13:02.415050 22508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:02.429888 22508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:02.433068 22508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:02.433367 22508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:02.434118 22508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:02.435015 22508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:02.435063 22508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:02.435132 22508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:02.435161 22508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:02.444742 22508 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:42575
I20260808 02:13:02.444779 22648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:42575 every 8 connection(s)
I20260808 02:13:02.445819 22508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:02.454344 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 22508
I20260808 02:13:02.454502 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:02.454792 19170 external_mini_cluster.cc:1428] Setting key 1ac8a2cd3b1054f8917913e3a2ce053a
Aug 08 02:13:02 dist-test-slave-9gcw krb5kdc[22395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:02.460808 22485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:02.447917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:58455 (local address 127.18.184.190:38515)
0808 02:13:02.448093 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:02.448098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:02.448937 (+ 839us) server_negotiation.cc:407] Connection header received
0808 02:13:02.450369 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:02.450372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:02.450421 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:02.450497 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:02.451778 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.452345 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.453611 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.453854 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.453939 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:02.456939 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:02.456943 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:02.456945 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:02.456968 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:02.458538 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:02.459106 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:02.459120 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:02.459121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:02.459199 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:02.459556 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:02.459560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:02.459562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:02.459773 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:02.459886 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:02.460541 (+ 655us) server_negotiation.cc:299] Negotiation successful
0808 02:13:02.460657 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:13:02.462173 22651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:02.448226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:38515 (local address 127.18.184.129:58455)
0808 02:13:02.448770 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:02.448818 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:02.450156 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:02.450671 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:02.450678 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:02.451118 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:02.451635 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:02.451650 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.452463 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.452466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:02.453498 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:02.453509 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.453998 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.454000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:02.454148 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:02.454898 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:02.454927 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:02.456765 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:02.458679 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:02.458685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:02.458702 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:02.458990 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:02.459303 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:02.459307 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:02.459308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:02.459458 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:02.459886 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:02.459895 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:02.460256 (+ 361us) client_negotiation.cc:769] Sending connection context
0808 02:13:02.460486 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 02:13:02.460907 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":114,"threads_started":1}
I20260808 02:13:02.463593 22649 heartbeater.cc:347] Connected to a master server at 127.18.184.190:38515
I20260808 02:13:02.463878 22649 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:02.464391 22649 heartbeater.cc:511] Master 127.18.184.190:38515 requested a full tablet report, sending...
I20260808 02:13:02.466013 22443 ts_manager.cc:194] Registered new tserver with Master: 9ab3451d830b463d985f8b20e8f00a42 (127.18.184.129:42575)
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:13:02.467154 22443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:58455
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:13:02.488332 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:38515
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:02.607364 22654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:02.607694 22654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:02.607776 22654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:02.611860 22654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:02.611968 22654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:02.612074 22654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:02.617550 22654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:38515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:02.618832 22654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:02.619979 22654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:02.630198 22662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:02.630221 22663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:02.630306 22654 server_base.cc:1034] running on GCE node
W20260808 02:13:02.630573 22665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:02.631096 22654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:02.631687 22654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:02.632862 22654 hybrid_clock.cc:648] HybridClock initialized: now 1786155182632847 us; error 27 us; skew 500 ppm
Aug 08 02:13:02 dist-test-slave-9gcw krb5kdc[22395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:02.635994 22654 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:13:02.636931 22654 webserver.cc:460] Webserver started at http://127.18.184.130:32817/ using document root <none> and password file <none>
I20260808 02:13:02.637641 22654 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:02.637734 22654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:02.637969 22654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:02.640070 22654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "d052a17061e94430addf5cdc71199457"
format_stamp: "Formatted at 2026-08-08 02:13:02 on dist-test-slave-9gcw"
server_key: "6a38546154ecd0337db1e8835066fea5"
server_key_iv: "13564d69a47b0618723fa0746f7ee405"
server_key_version: "encryptionkey@0"
I20260808 02:13:02.640626 22654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "d052a17061e94430addf5cdc71199457"
format_stamp: "Formatted at 2026-08-08 02:13:02 on dist-test-slave-9gcw"
server_key: "6a38546154ecd0337db1e8835066fea5"
server_key_iv: "13564d69a47b0618723fa0746f7ee405"
server_key_version: "encryptionkey@0"
I20260808 02:13:02.644335 22654 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 02:13:02.646780 22672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:02.647980 22654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 02:13:02.648123 22654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "d052a17061e94430addf5cdc71199457"
format_stamp: "Formatted at 2026-08-08 02:13:02 on dist-test-slave-9gcw"
server_key: "6a38546154ecd0337db1e8835066fea5"
server_key_iv: "13564d69a47b0618723fa0746f7ee405"
server_key_version: "encryptionkey@0"
I20260808 02:13:02.648240 22654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:02.663111 22654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:02.666508 22654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:02.666674 22654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:02.667256 22654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:02.668113 22654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:02.668162 22654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:02.668205 22654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:02.668220 22654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:02.677567 22654 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:44357
I20260808 02:13:02.677572 22793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:44357 every 8 connection(s)
I20260808 02:13:02.678576 22654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:02.685829 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 22654
I20260808 02:13:02.685981 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:02.686304 19170 external_mini_cluster.cc:1428] Setting key 40127e4b7ec6fa19579bc2a97a4cd48f
Aug 08 02:13:02 dist-test-slave-9gcw krb5kdc[22395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:02.693991 22485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:02.680525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:54747 (local address 127.18.184.190:38515)
0808 02:13:02.680658 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:02.680662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:02.681553 (+ 891us) server_negotiation.cc:407] Connection header received
0808 02:13:02.683106 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:02.683109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:02.683170 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:02.683249 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:02.684465 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.685014 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.686271 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.686570 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.686678 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:02.689662 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:02.689665 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:02.689668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:02.689695 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:02.691453 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:02.692118 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:02.692121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:02.692122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:02.692217 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:02.692622 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:02.692624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:02.692626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:02.692831 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:02.692915 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:02.693680 (+ 765us) server_negotiation.cc:299] Negotiation successful
0808 02:13:02.693860 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:13:02.695317 22796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:02.680918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:38515 (local address 127.18.184.130:54747)
0808 02:13:02.681410 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:02.681453 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:02.682888 (+ 1435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:02.683421 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:02.683428 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:02.683845 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:02.684322 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:02.684337 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.685186 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.685189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:02.686152 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:02.686162 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.686726 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.686730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:02.686922 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:02.687595 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:02.687633 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:02.689493 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:02.691566 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:02.691574 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:02.691596 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:02.691994 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:02.692311 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:02.692314 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:02.692318 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:02.692522 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:02.692939 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:02.692949 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:02.693383 (+ 434us) client_negotiation.cc:769] Sending connection context
0808 02:13:02.693619 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 02:13:02.694039 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":160,"threads_started":1}
I20260808 02:13:02.696626 22794 heartbeater.cc:347] Connected to a master server at 127.18.184.190:38515
I20260808 02:13:02.696938 22794 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:02.697546 22794 heartbeater.cc:511] Master 127.18.184.190:38515 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:13:02.698621 22443 ts_manager.cc:194] Registered new tserver with Master: d052a17061e94430addf5cdc71199457 (127.18.184.130:44357)
I20260808 02:13:02.699278 22443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:54747
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:13:02.718051 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:38515
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:02.835407 22799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:02.835690 22799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:02.835736 22799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:02.839820 22799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:02.839886 22799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:02.839944 22799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:02.845252 22799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:38515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:02.846243 22799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:02.847193 22799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:02.856832 22807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:02.856843 22804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:02.856843 22805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:02.857033 22799 server_base.cc:1034] running on GCE node
I20260808 02:13:02.857479 22799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:02.858176 22799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:02.859380 22799 hybrid_clock.cc:648] HybridClock initialized: now 1786155182859367 us; error 27 us; skew 500 ppm
Aug 08 02:13:02 dist-test-slave-9gcw krb5kdc[22395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:02.862461 22799 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:13:02.863330 22799 webserver.cc:460] Webserver started at http://127.18.184.131:38511/ using document root <none> and password file <none>
I20260808 02:13:02.863858 22799 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:02.863920 22799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:02.864090 22799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:02.866266 22799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "aa93c8a4d74e46aa861acbd38f5f79c5"
format_stamp: "Formatted at 2026-08-08 02:13:02 on dist-test-slave-9gcw"
server_key: "7650315401f91bd0446b62f4298a7203"
server_key_iv: "55e7de2a9b5f8e12b41b881ac3b8dedf"
server_key_version: "encryptionkey@0"
I20260808 02:13:02.866815 22799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "aa93c8a4d74e46aa861acbd38f5f79c5"
format_stamp: "Formatted at 2026-08-08 02:13:02 on dist-test-slave-9gcw"
server_key: "7650315401f91bd0446b62f4298a7203"
server_key_iv: "55e7de2a9b5f8e12b41b881ac3b8dedf"
server_key_version: "encryptionkey@0"
I20260808 02:13:02.870642 22799 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:13:02.873421 22816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:02.874760 22799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:02.874902 22799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "aa93c8a4d74e46aa861acbd38f5f79c5"
format_stamp: "Formatted at 2026-08-08 02:13:02 on dist-test-slave-9gcw"
server_key: "7650315401f91bd0446b62f4298a7203"
server_key_iv: "55e7de2a9b5f8e12b41b881ac3b8dedf"
server_key_version: "encryptionkey@0"
I20260808 02:13:02.875023 22799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:02.887698 22799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:02.891258 22799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:02.891434 22799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:02.892020 22799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:02.892860 22799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:02.892908 22799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:02.892949 22799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:02.892971 22799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:02.902500 22799 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:36103
I20260808 02:13:02.902559 22939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:36103 every 8 connection(s)
I20260808 02:13:02.903512 22799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:02.905954 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 22799
I20260808 02:13:02.906085 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:02.906445 19170 external_mini_cluster.cc:1428] Setting key 5c7a1b7e2bd331fa6e4148de03a05829
Aug 08 02:13:02 dist-test-slave-9gcw krb5kdc[22395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:02.918576 22485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:02.905624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:34523 (local address 127.18.184.190:38515)
0808 02:13:02.905771 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:02.905774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:02.907102 (+ 1328us) server_negotiation.cc:407] Connection header received
0808 02:13:02.908901 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:02.908904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:02.908963 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:02.909066 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:02.910402 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.910938 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.912085 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.912308 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.912409 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:02.914981 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:02.914984 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:02.914986 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:02.915015 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:02.916461 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:02.916959 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:02.916962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:02.916963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:02.917048 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:02.917411 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:02.917414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:02.917415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:02.917609 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:02.917694 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:02.918322 (+ 628us) server_negotiation.cc:299] Negotiation successful
0808 02:13:02.918435 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:13:02.919528 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:02.906254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:38515 (local address 127.18.184.131:34523)
0808 02:13:02.906948 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:02.907009 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:02.908672 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:02.909294 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:02.909301 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:02.909764 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:02.910262 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:02.910276 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.911066 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.911070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:02.911960 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:02.911969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.912466 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.912470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:02.912648 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:02.913283 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:02.913300 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:02.914814 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:02.916581 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:02.916585 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:02.916597 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:02.916861 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:02.917131 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:02.917133 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:02.917134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:02.917325 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:02.917752 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:02.917757 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:02.918075 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 02:13:02.918253 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 02:13:02.918522 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":161,"threads_started":1}
I20260808 02:13:02.920800 22940 heartbeater.cc:347] Connected to a master server at 127.18.184.190:38515
I20260808 02:13:02.921044 22940 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:02.921651 22940 heartbeater.cc:511] Master 127.18.184.190:38515 requested a full tablet report, sending...
I20260808 02:13:02.922791 22442 ts_manager.cc:194] Registered new tserver with Master: aa93c8a4d74e46aa861acbd38f5f79c5 (127.18.184.131:36103)
I20260808 02:13:02.923373 22442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:34523
I20260808 02:13:02.932686 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:13:02 dist-test-slave-9gcw krb5kdc[22395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155182, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 02:13:02 dist-test-slave-9gcw krb5kdc[22395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155182, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:02.960026 22485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:02.951397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58314 (local address 127.18.184.190:38515)
0808 02:13:02.951597 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:02.951601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:02.951771 (+ 170us) server_negotiation.cc:407] Connection header received
0808 02:13:02.951915 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:02.951918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:02.951964 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:02.952046 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:02.952817 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.953402 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.954245 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:02.954449 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:02.954526 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:02.957073 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:02.957076 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:02.957078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:02.957103 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:02.958597 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:02.958967 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:02.958969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:02.958971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:02.959046 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:02.959267 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:02.959269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:02.959270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:02.959492 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:02.959597 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:02.959799 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 02:13:02.959897 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260808 02:13:02.960448 22442 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58314
I20260808 02:13:02.961454 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 22508
I20260808 02:13:02.969441 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 22654
I20260808 02:13:02.977053 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 22799
I20260808 02:13:02.984931 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 22408
2026-08-08T02:13:02Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2271 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:03 dist-test-slave-9gcw krb5kdc[22955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:03 dist-test-slave-9gcw krb5kdc[22955](info): set up 2 sockets
Aug 08 02:13:03 dist-test-slave-9gcw krb5kdc[22955](info): commencing operation
krb5kdc: starting...
W20260808 02:13:04.082634 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:13:04 dist-test-slave-9gcw krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:13:04.220530 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:45687
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:45687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:04.335963 22968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:04.336292 22968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:04.336385 22968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:04.340454 22968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:04.340546 22968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:04.340584 22968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:04.340637 22968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:04.340670 22968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:04.346040 22968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:45687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:45687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:04.347147 22968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:04.348209 22968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:04.356722 22977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:04.356880 22968 server_base.cc:1034] running on GCE node
W20260808 02:13:04.356815 22973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:04.356838 22974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:04.357633 22968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:04.358753 22968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:04.359917 22968 hybrid_clock.cc:648] HybridClock initialized: now 1786155184359906 us; error 29 us; skew 500 ppm
Aug 08 02:13:04 dist-test-slave-9gcw krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:04.362947 22968 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:13:04.363866 22968 webserver.cc:460] Webserver started at http://127.18.184.190:41089/ using document root <none> and password file <none>
I20260808 02:13:04.364475 22968 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:04.364540 22968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:04.364792 22968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:04.366942 22968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "050cbd0e81ff435fb6f342e407593a38"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-9gcw"
server_key: "9dd1b81e5fd0e9b6e88c5a170e153a77"
server_key_iv: "089c779b26fa516b035ac69320c7a38c"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.367515 22968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "050cbd0e81ff435fb6f342e407593a38"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-9gcw"
server_key: "9dd1b81e5fd0e9b6e88c5a170e153a77"
server_key_iv: "089c779b26fa516b035ac69320c7a38c"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.371341 22968 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:13:04.373840 22985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.375120 22968 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 02:13:04.375284 22968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "050cbd0e81ff435fb6f342e407593a38"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-9gcw"
server_key: "9dd1b81e5fd0e9b6e88c5a170e153a77"
server_key_iv: "089c779b26fa516b035ac69320c7a38c"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.375396 22968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:04.398660 22968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:04.401615 22968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:04.401826 22968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:04.409284 22968 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:45687
I20260808 02:13:04.409309 23039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:45687 every 8 connection(s)
I20260808 02:13:04.410385 22968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:04.413257 23040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:04.418150 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 22968
I20260808 02:13:04.418280 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:04.418602 19170 external_mini_cluster.cc:1428] Setting key b7fb923475fac39cc2a6703d243f105d
I20260808 02:13:04.419358 23040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38: Bootstrap starting.
I20260808 02:13:04.421962 23040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:04.422763 23040 log.cc:824] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:04.424817 23040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38: No bootstrap required, opened a new log
Aug 08 02:13:04 dist-test-slave-9gcw krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:04.429243 23040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050cbd0e81ff435fb6f342e407593a38" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45687 } }
I20260808 02:13:04.429582 23040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:04.429656 23040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 050cbd0e81ff435fb6f342e407593a38, State: Initialized, Role: FOLLOWER
I20260808 02:13:04.430168 23040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050cbd0e81ff435fb6f342e407593a38" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45687 } }
I20260808 02:13:04.430301 23040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:04.430392 23040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:04.430491 23040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:04.431560 23040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050cbd0e81ff435fb6f342e407593a38" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45687 } }
I20260808 02:13:04.431962 23040 leader_election.cc:302] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 050cbd0e81ff435fb6f342e407593a38; no voters:
I20260808 02:13:04.432374 23040 leader_election.cc:288] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:04.432468 23046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:04.432713 23046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [term 1 LEADER]: Becoming Leader. State: Replica: 050cbd0e81ff435fb6f342e407593a38, State: Running, Role: LEADER
I20260808 02:13:04.433081 23046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050cbd0e81ff435fb6f342e407593a38" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45687 } }
I20260808 02:13:04.433878 23040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:04.434816 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:04.420156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59758 (local address 127.18.184.190:45687)
0808 02:13:04.420637 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:04.420652 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:04.420691 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:13:04.421436 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:04.421455 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:04.421803 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:04.422145 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:04.423073 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.424055 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.424980 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.425559 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.425734 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:04.428852 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:04.428859 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:04.428877 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:04.428911 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:04.431112 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.431801 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.431810 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.431816 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.431960 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.432292 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.432294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.432295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.432906 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:04.433088 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:04.433447 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 02:13:04.433762 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":135,"threads_started":1}
I20260808 02:13:04.434831 23047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "050cbd0e81ff435fb6f342e407593a38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050cbd0e81ff435fb6f342e407593a38" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45687 } } }
I20260808 02:13:04.434963 23047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:04.435276 23049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 050cbd0e81ff435fb6f342e407593a38. Latest consensus state: current_term: 1 leader_uuid: "050cbd0e81ff435fb6f342e407593a38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050cbd0e81ff435fb6f342e407593a38" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45687 } } }
I20260808 02:13:04.435374 23049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:04.435410 23056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:04.438673 23056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:04.444573 23056 catalog_manager.cc:1347] Generated new cluster ID: aca2ae0ee5324ff1a95a479a2a020a97
I20260808 02:13:04.444651 23056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:04.461124 23056 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:04.462301 23056 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:04.471047 23056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 050cbd0e81ff435fb6f342e407593a38: Generated new TSK 0
I20260808 02:13:04.471787 23056 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:13:04.508846 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:45687
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:04.645848 23069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:04.646159 23069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:04.646211 23069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:04.650372 23069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:04.650446 23069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:04.650507 23069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:04.656291 23069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:45687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:04.657437 23069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:04.658447 23069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:04.668350 23074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:04.668503 23075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:04.668349 23078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:04.668644 23069 server_base.cc:1034] running on GCE node
I20260808 02:13:04.669054 23069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:04.669735 23069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:04.670930 23069 hybrid_clock.cc:648] HybridClock initialized: now 1786155184670909 us; error 37 us; skew 500 ppm
Aug 08 02:13:04 dist-test-slave-9gcw krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:04.673799 23069 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:13:04.674690 23069 webserver.cc:460] Webserver started at http://127.18.184.129:34017/ using document root <none> and password file <none>
I20260808 02:13:04.675267 23069 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:04.675370 23069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:04.675603 23069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:04.677747 23069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "008e36bfd4354c46a5d2aa7d995c09a8"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-9gcw"
server_key: "f727e1443dbcc36456371bc94afe9e11"
server_key_iv: "efdabe03b9a0d8bb4d58a5edc7bd5bb5"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.678330 23069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "008e36bfd4354c46a5d2aa7d995c09a8"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-9gcw"
server_key: "f727e1443dbcc36456371bc94afe9e11"
server_key_iv: "efdabe03b9a0d8bb4d58a5edc7bd5bb5"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.682287 23069 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 02:13:04.684903 23085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.686306 23069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:04.686460 23069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "008e36bfd4354c46a5d2aa7d995c09a8"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-9gcw"
server_key: "f727e1443dbcc36456371bc94afe9e11"
server_key_iv: "efdabe03b9a0d8bb4d58a5edc7bd5bb5"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.686591 23069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:04.731266 23069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:04.734905 23069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:04.735138 23069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:04.735833 23069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:04.736712 23069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:04.736765 23069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.736810 23069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:04.736876 23069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.747267 23069 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:43755
I20260808 02:13:04.747335 23210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:43755 every 8 connection(s)
I20260808 02:13:04.748301 23069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:04.757411 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 23069
I20260808 02:13:04.757527 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:04.757822 19170 external_mini_cluster.cc:1428] Setting key dd0dcb6e1796e94e7c1d31e360d4b43b
Aug 08 02:13:04 dist-test-slave-9gcw krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:04.764568 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:04.750390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:41075 (local address 127.18.184.190:45687)
0808 02:13:04.750565 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:04.750571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:04.751721 (+ 1150us) server_negotiation.cc:407] Connection header received
0808 02:13:04.753633 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:04.753636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:04.753699 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:04.753783 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:04.755049 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.755616 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.756813 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.757086 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.757230 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:04.760315 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:04.760321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:04.760328 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:04.760366 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:04.762122 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.762668 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.762671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.762673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.762772 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.763146 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.763152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.763156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.763447 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:04.763557 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:04.764094 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 02:13:04.764284 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:13:04.765380 23213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:04.750907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:45687 (local address 127.18.184.129:41075)
0808 02:13:04.751529 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:04.751593 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:04.753391 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:04.753959 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:04.753966 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:04.754388 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:04.754892 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:04.754907 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.755758 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.755762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:04.756650 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:04.756659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.757381 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.757384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:04.757646 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:04.758363 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:04.758382 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:04.760119 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:04.762253 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:04.762260 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:04.762278 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:04.762559 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:04.762887 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:04.762890 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:04.762892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:04.763043 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:04.763557 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:04.763562 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:04.763836 (+ 274us) client_negotiation.cc:769] Sending connection context
0808 02:13:04.764006 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 02:13:04.764286 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":202,"threads_started":1}
I20260808 02:13:04.766777 23211 heartbeater.cc:347] Connected to a master server at 127.18.184.190:45687
I20260808 02:13:04.767067 23211 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:04.767715 23211 heartbeater.cc:511] Master 127.18.184.190:45687 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:13:04.769639 23002 ts_manager.cc:194] Registered new tserver with Master: 008e36bfd4354c46a5d2aa7d995c09a8 (127.18.184.129:43755)
I20260808 02:13:04.770874 23002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:41075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:13:04.788763 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:45687
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:04.905318 23216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:04.905639 23216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:04.905720 23216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:04.909900 23216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:04.909991 23216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:04.910082 23216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:04.915684 23216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:45687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:04.916841 23216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:04.918007 23216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:04.929195 23223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:04.929175 23222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:04.929515 23225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:04.930112 23216 server_base.cc:1034] running on GCE node
I20260808 02:13:04.930634 23216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:04.931540 23216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:04.932732 23216 hybrid_clock.cc:648] HybridClock initialized: now 1786155184932720 us; error 62 us; skew 500 ppm
Aug 08 02:13:04 dist-test-slave-9gcw krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:04.935911 23216 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:13:04.936739 23216 webserver.cc:460] Webserver started at http://127.18.184.130:46113/ using document root <none> and password file <none>
I20260808 02:13:04.937335 23216 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:04.937398 23216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:04.937566 23216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:04.939589 23216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "ace07497df3e4c3e9d4b08a1ed1959ae"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-9gcw"
server_key: "eedac8841de223687a5e8cd10a0aaa1b"
server_key_iv: "46c95492910df79fe7ae03060e03df05"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.940097 23216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "ace07497df3e4c3e9d4b08a1ed1959ae"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-9gcw"
server_key: "eedac8841de223687a5e8cd10a0aaa1b"
server_key_iv: "46c95492910df79fe7ae03060e03df05"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.943776 23216 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 02:13:04.946228 23233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.947454 23216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 02:13:04.947574 23216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "ace07497df3e4c3e9d4b08a1ed1959ae"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-9gcw"
server_key: "eedac8841de223687a5e8cd10a0aaa1b"
server_key_iv: "46c95492910df79fe7ae03060e03df05"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.947675 23216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:04.959906 23216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:04.963291 23216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:04.963497 23216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:04.964128 23216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:04.965032 23216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:04.965080 23216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.965147 23216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:04.965214 23216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.974630 23216 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:43087
I20260808 02:13:04.974648 23356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:43087 every 8 connection(s)
I20260808 02:13:04.975590 23216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:04.976325 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 23216
I20260808 02:13:04.976446 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:04.976713 19170 external_mini_cluster.cc:1428] Setting key c4f0e2ae37c809425074a6fb20208031
Aug 08 02:13:04 dist-test-slave-9gcw krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
I20260808 02:13:04.990172 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:04.977671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:47527 (local address 127.18.184.190:45687)
0808 02:13:04.977815 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:04.977820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:04.978665 (+ 845us) server_negotiation.cc:407] Connection header received
0808 02:13:04.980195 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:04.980198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:04.980270 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:04.980346 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:04.981684 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.982212 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.983360 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.983594 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.983701 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:04.986285 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:04.986289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:04.986292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:04.986321 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:04.988011 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.988508 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.988511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.988512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.988586 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.988876 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.988878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.988879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.989053 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:04.989127 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:04.989763 (+ 636us) server_negotiation.cc:299] Negotiation successful
0808 02:13:04.989993 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:13:04.991312 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:04.977989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:45687 (local address 127.18.184.130:47527)
0808 02:13:04.978505 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:04.978556 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:04.979960 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:04.980595 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:04.980602 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:04.981000 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:04.981545 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:04.981559 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.982347 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.982350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:04.983234 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:04.983245 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.983759 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.983763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:04.983933 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:04.984576 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:04.984592 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:04.986065 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:04.988126 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:04.988131 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:04.988144 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:04.988424 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:04.988664 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:04.988666 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:04.988667 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:04.988799 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:04.989141 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:04.989189 (+ 48us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:04.989492 (+ 303us) client_negotiation.cc:769] Sending connection context
0808 02:13:04.989695 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 02:13:04.990069 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":148,"threads_started":1}
I20260808 02:13:04.992635 23357 heartbeater.cc:347] Connected to a master server at 127.18.184.190:45687
I20260808 02:13:04.992893 23357 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:04.993469 23357 heartbeater.cc:511] Master 127.18.184.190:45687 requested a full tablet report, sending...
I20260808 02:13:04.994699 23002 ts_manager.cc:194] Registered new tserver with Master: ace07497df3e4c3e9d4b08a1ed1959ae (127.18.184.130:43087)
I20260808 02:13:04.995608 23002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:47527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:13:05.008837 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:45687
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:05.130972 23362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:05.131275 23362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:05.131325 23362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:05.135795 23362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:05.135877 23362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:05.135942 23362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:05.141690 23362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:45687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:05.142896 23362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:05.143941 23362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:05.154466 23370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:05.154579 23368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:05.154731 23362 server_base.cc:1034] running on GCE node
W20260808 02:13:05.154466 23367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:05.155484 23362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:05.156289 23362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:05.157469 23362 hybrid_clock.cc:648] HybridClock initialized: now 1786155185157453 us; error 30 us; skew 500 ppm
Aug 08 02:13:05 dist-test-slave-9gcw krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155185, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:05.160352 23362 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:13:05.161381 23362 webserver.cc:460] Webserver started at http://127.18.184.131:45761/ using document root <none> and password file <none>
I20260808 02:13:05.161949 23362 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:05.162037 23362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:05.162266 23362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:05.164361 23362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "85742c3f900546d9b9712de7b6330239"
format_stamp: "Formatted at 2026-08-08 02:13:05 on dist-test-slave-9gcw"
server_key: "11c5759d1b5dc667d3a4ae2d06ab6cd8"
server_key_iv: "1129a91f2976452ccd3526d872954ac0"
server_key_version: "encryptionkey@0"
I20260808 02:13:05.164994 23362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "85742c3f900546d9b9712de7b6330239"
format_stamp: "Formatted at 2026-08-08 02:13:05 on dist-test-slave-9gcw"
server_key: "11c5759d1b5dc667d3a4ae2d06ab6cd8"
server_key_iv: "1129a91f2976452ccd3526d872954ac0"
server_key_version: "encryptionkey@0"
I20260808 02:13:05.168862 23362 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 02:13:05.171314 23377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:05.172520 23362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:05.172653 23362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "85742c3f900546d9b9712de7b6330239"
format_stamp: "Formatted at 2026-08-08 02:13:05 on dist-test-slave-9gcw"
server_key: "11c5759d1b5dc667d3a4ae2d06ab6cd8"
server_key_iv: "1129a91f2976452ccd3526d872954ac0"
server_key_version: "encryptionkey@0"
I20260808 02:13:05.172765 23362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:05.195502 23362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:05.198917 23362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:05.199133 23362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:05.199775 23362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:05.200666 23362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:05.200735 23362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:05.200807 23362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:05.200850 23362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:05.210071 23362 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:44317
I20260808 02:13:05.210095 23501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:44317 every 8 connection(s)
I20260808 02:13:05.210969 23362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:05.217234 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 23362
I20260808 02:13:05.217372 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:05.217651 19170 external_mini_cluster.cc:1428] Setting key 3bef5fb73177ec4df98e84072c8146f2
Aug 08 02:13:05 dist-test-slave-9gcw krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155185, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:05.224654 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:05.212897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:58817 (local address 127.18.184.190:45687)
0808 02:13:05.213052 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:05.213056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:05.213928 (+ 872us) server_negotiation.cc:407] Connection header received
0808 02:13:05.215267 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:05.215270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:05.215320 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:05.215406 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:05.216579 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:05.217127 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:05.218601 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:05.218823 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:05.218935 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:05.221557 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:05.221560 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:05.221562 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:05.221588 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:05.222868 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:05.223350 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:05.223353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:05.223354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:05.223427 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:05.223712 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:05.223714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:05.223715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:05.223893 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:05.223969 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:05.224428 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 02:13:05.224529 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:13:05.225536 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:05.213297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:45687 (local address 127.18.184.131:58817)
0808 02:13:05.213774 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:05.213817 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:05.215073 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:05.215570 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:05.215577 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:05.215973 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:05.216449 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:05.216463 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:05.217286 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:05.217289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:05.218435 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:05.218448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:05.218979 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:05.218983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:05.219180 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:05.219881 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:05.219904 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:05.221393 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:05.222974 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:05.222979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:05.222992 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:05.223262 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:05.223501 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:05.223504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:05.223505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:05.223639 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:05.223981 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:05.223986 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:05.224220 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 02:13:05.224385 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 02:13:05.224642 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":111,"threads_started":1}
I20260808 02:13:05.226774 23502 heartbeater.cc:347] Connected to a master server at 127.18.184.190:45687
I20260808 02:13:05.227020 23502 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:05.227516 23502 heartbeater.cc:511] Master 127.18.184.190:45687 requested a full tablet report, sending...
I20260808 02:13:05.228545 23002 ts_manager.cc:194] Registered new tserver with Master: 85742c3f900546d9b9712de7b6330239 (127.18.184.131:44317)
I20260808 02:13:05.229111 23002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:58817
I20260808 02:13:05.231547 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:13:05 dist-test-slave-9gcw krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:13:05 dist-test-slave-9gcw krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.129@KRBTEST.COM
I20260808 02:13:05.259421 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:05.249459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43064 (local address 127.18.184.129:43755)
0808 02:13:05.249758 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:05.249773 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:05.249800 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:13:05.249988 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:05.249995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:05.250210 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:05.250344 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:05.251186 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:05.252129 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:05.252960 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:05.253404 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:05.253488 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:05.255894 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:05.255905 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:05.255918 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:05.255946 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:05.257642 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:05.258111 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:05.258115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:05.258117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:05.258197 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:05.258453 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:05.258455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:05.258456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:05.258693 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:05.258865 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:05.259098 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 02:13:05.259233 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":87,"threads_started":1}
Aug 08 02:13:05 dist-test-slave-9gcw krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:05.270838 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:05.262578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59768 (local address 127.18.184.190:45687)
0808 02:13:05.262716 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:05.262723 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:05.262765 (+ 42us) server_negotiation.cc:407] Connection header received
0808 02:13:05.262919 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:05.262922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:05.262973 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:05.263059 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:05.263682 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:05.264208 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:05.265007 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:05.265301 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:05.265407 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:05.267801 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:05.267805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:05.267816 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:05.267853 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:05.269080 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:05.269522 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:05.269525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:05.269527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:05.269614 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:05.269882 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:05.269887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:05.269888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:05.270130 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:05.270272 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:05.270532 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 02:13:05.270655 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 02:13:05.271289 23002 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59768
I20260808 02:13:05.271943 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 23069
I20260808 02:13:05.282202 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 23216
I20260808 02:13:05.289458 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 23362
I20260808 02:13:05.296980 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 22968
2026-08-08T02:13:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2310 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:05 dist-test-slave-9gcw krb5kdc[23514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:05 dist-test-slave-9gcw krb5kdc[23514](info): set up 2 sockets
Aug 08 02:13:05 dist-test-slave-9gcw krb5kdc[23514](info): commencing operation
krb5kdc: starting...
W20260808 02:13:06.449513 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:13:06 dist-test-slave-9gcw krb5kdc[23514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:13:06.588976 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:44617
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:44617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:06.704604 23529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:06.704952 23529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:06.705039 23529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:06.709118 23529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:06.709244 23529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:06.709283 23529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:06.709333 23529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:06.709374 23529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:06.714820 23529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:44617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:44617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:06.716156 23529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:06.717262 23529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:06.725981 23536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:06.725996 23535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:06.726143 23529 server_base.cc:1034] running on GCE node
W20260808 02:13:06.725955 23538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:06.726848 23529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:06.728020 23529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:06.729226 23529 hybrid_clock.cc:648] HybridClock initialized: now 1786155186729218 us; error 38 us; skew 500 ppm
Aug 08 02:13:06 dist-test-slave-9gcw krb5kdc[23514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155186, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:06.737805 23529 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:13:06.738716 23529 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:06.738819 23529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:06.739079 23529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:06.741329 23529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "e01d57b362c246519b53828a13a12398"
format_stamp: "Formatted at 2026-08-08 02:13:06 on dist-test-slave-9gcw"
server_key: "884b4186f91b5f9f172644993f3d75a1"
server_key_iv: "eb7050e11755cab0f3890d88123c007f"
server_key_version: "encryptionkey@0"
I20260808 02:13:06.741937 23529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "e01d57b362c246519b53828a13a12398"
format_stamp: "Formatted at 2026-08-08 02:13:06 on dist-test-slave-9gcw"
server_key: "884b4186f91b5f9f172644993f3d75a1"
server_key_iv: "eb7050e11755cab0f3890d88123c007f"
server_key_version: "encryptionkey@0"
I20260808 02:13:06.745790 23529 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:13:06.748283 23545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:06.749738 23529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:06.749889 23529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "e01d57b362c246519b53828a13a12398"
format_stamp: "Formatted at 2026-08-08 02:13:06 on dist-test-slave-9gcw"
server_key: "884b4186f91b5f9f172644993f3d75a1"
server_key_iv: "eb7050e11755cab0f3890d88123c007f"
server_key_version: "encryptionkey@0"
I20260808 02:13:06.750012 23529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:06.788367 23529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:06.791760 23529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:06.792028 23529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:06.799706 23529 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:44617
I20260808 02:13:06.800017 23602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:44617 every 8 connection(s)
I20260808 02:13:06.800447 23529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:06.803350 23603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:06.807574 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 23529
I20260808 02:13:06.807752 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:06.808017 19170 external_mini_cluster.cc:1428] Setting key a2616bacd33175b53d0c6eb315175f8b
I20260808 02:13:06.809566 23603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398: Bootstrap starting.
I20260808 02:13:06.812002 23603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:06.812866 23603 log.cc:824] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:06.815259 23603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398: No bootstrap required, opened a new log
Aug 08 02:13:06 dist-test-slave-9gcw krb5kdc[23514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:06.820182 23603 raft_consensus.cc:348] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e01d57b362c246519b53828a13a12398" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44617 } }
I20260808 02:13:06.820479 23603 raft_consensus.cc:374] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:06.820530 23603 raft_consensus.cc:729] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01d57b362c246519b53828a13a12398, State: Initialized, Role: FOLLOWER
I20260808 02:13:06.821058 23603 consensus_queue.cc:260] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e01d57b362c246519b53828a13a12398" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44617 } }
I20260808 02:13:06.821277 23603 raft_consensus.cc:388] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:06.821352 23603 raft_consensus.cc:482] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:06.821457 23603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:06.822443 23603 raft_consensus.cc:504] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e01d57b362c246519b53828a13a12398" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44617 } }
I20260808 02:13:06.822841 23603 leader_election.cc:302] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e01d57b362c246519b53828a13a12398; no voters:
I20260808 02:13:06.823254 23603 leader_election.cc:288] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:06.823364 23608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:06.823575 23608 raft_consensus.cc:686] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [term 1 LEADER]: Becoming Leader. State: Replica: e01d57b362c246519b53828a13a12398, State: Running, Role: LEADER
I20260808 02:13:06.823966 23608 consensus_queue.cc:237] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e01d57b362c246519b53828a13a12398" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44617 } }
I20260808 02:13:06.824383 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:06.809564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54442 (local address 127.18.184.190:44617)
0808 02:13:06.810089 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:06.810108 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:06.810148 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:13:06.810848 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:06.810865 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:06.811202 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:06.811483 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:06.812495 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:06.813840 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:06.815018 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:06.815544 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:06.815698 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:06.818537 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:06.818546 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:06.818567 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:06.818609 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:06.820881 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:06.821439 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:06.821445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:06.821451 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:06.821617 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:06.821941 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:06.821945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:06.821947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:06.822380 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:06.822597 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:06.822940 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 02:13:06.823317 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":155,"threads_started":1}
I20260808 02:13:06.824606 23603 sys_catalog.cc:564] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:06.825680 23609 sys_catalog.cc:455] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e01d57b362c246519b53828a13a12398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e01d57b362c246519b53828a13a12398" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44617 } } }
I20260808 02:13:06.825841 23609 sys_catalog.cc:458] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:06.825846 23611 sys_catalog.cc:455] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e01d57b362c246519b53828a13a12398. Latest consensus state: current_term: 1 leader_uuid: "e01d57b362c246519b53828a13a12398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e01d57b362c246519b53828a13a12398" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44617 } } }
I20260808 02:13:06.825950 23611 sys_catalog.cc:458] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:06.827382 23614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:06.830504 23614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:06.836611 23614 catalog_manager.cc:1347] Generated new cluster ID: fc3d30fc44ac450a953f4badfb06090c
I20260808 02:13:06.836724 23614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:06.853469 23614 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:06.854830 23614 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:06.871235 23614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398: Generated new TSK 0
I20260808 02:13:06.872213 23614 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:13:06.913064 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:44617
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:07.034814 23632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:07.035104 23632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:07.035151 23632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:07.039237 23632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:07.039304 23632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:07.039368 23632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:07.044773 23632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:44617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:07.045944 23632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:07.046949 23632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:13:07.057222 23632 server_base.cc:1034] running on GCE node
W20260808 02:13:07.057129 23640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:07.057137 23637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:07.057176 23638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:07.058090 23632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:07.058861 23632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:07.060096 23632 hybrid_clock.cc:648] HybridClock initialized: now 1786155187060065 us; error 51 us; skew 500 ppm
Aug 08 02:13:07 dist-test-slave-9gcw krb5kdc[23514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155187, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:07.063337 23632 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:13:07.064129 23632 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:07.064230 23632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:07.064487 23632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:07.066871 23632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "f7bd2175ba344aff814ff98cd21e5804"
format_stamp: "Formatted at 2026-08-08 02:13:07 on dist-test-slave-9gcw"
server_key: "7745b95f33110d9bd0d7c989641a0ac9"
server_key_iv: "af82a3504c98b31f00762fda1fe79b39"
server_key_version: "encryptionkey@0"
I20260808 02:13:07.067516 23632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "f7bd2175ba344aff814ff98cd21e5804"
format_stamp: "Formatted at 2026-08-08 02:13:07 on dist-test-slave-9gcw"
server_key: "7745b95f33110d9bd0d7c989641a0ac9"
server_key_iv: "af82a3504c98b31f00762fda1fe79b39"
server_key_version: "encryptionkey@0"
I20260808 02:13:07.071556 23632 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:13:07.074524 23646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:07.075783 23632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:07.075925 23632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "f7bd2175ba344aff814ff98cd21e5804"
format_stamp: "Formatted at 2026-08-08 02:13:07 on dist-test-slave-9gcw"
server_key: "7745b95f33110d9bd0d7c989641a0ac9"
server_key_iv: "af82a3504c98b31f00762fda1fe79b39"
server_key_version: "encryptionkey@0"
I20260808 02:13:07.076047 23632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:07.087718 23632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:07.091307 23632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:07.091562 23632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:07.092270 23632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:07.093375 23632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:07.093459 23632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:07.093549 23632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:07.093621 23632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:07.104671 23632 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:38803
I20260808 02:13:07.104697 23768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:38803 every 8 connection(s)
I20260808 02:13:07.105525 23632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:07.111091 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 23632
I20260808 02:13:07.111240 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:07.111599 19170 external_mini_cluster.cc:1428] Setting key 5d6f9375193b27b1fafde3a34e3020e3
Aug 08 02:13:07 dist-test-slave-9gcw krb5kdc[23514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155187, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:07.121178 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.107588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:54135 (local address 127.18.184.190:44617)
0808 02:13:07.107773 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:07.107778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:07.108715 (+ 937us) server_negotiation.cc:407] Connection header received
0808 02:13:07.110080 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:07.110084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:07.110134 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:07.110212 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:07.111835 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.112480 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.114150 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.114404 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.114492 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.117373 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:07.117377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:07.117379 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:07.117403 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:07.118978 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:07.119519 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:07.119528 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:07.119529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:07.119609 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:07.119981 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:07.119985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:07.119986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:07.120174 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:07.120294 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:07.120830 (+ 536us) server_negotiation.cc:299] Negotiation successful
0808 02:13:07.120968 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:13:07.122031 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.108035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:44617 (local address 127.18.184.129:54135)
0808 02:13:07.108545 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:07.108590 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:07.109873 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:07.110384 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:07.110390 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:07.110869 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:07.111593 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.111616 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.112624 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.112628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.114017 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.114033 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.114565 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.114572 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.114829 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.115602 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:07.115619 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:07.117197 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:07.119113 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:07.119119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:07.119135 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:07.119410 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:07.119702 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:07.119706 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:07.119708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:07.119864 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:07.120294 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:07.120302 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:07.120560 (+ 258us) client_negotiation.cc:769] Sending connection context
0808 02:13:07.120768 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 02:13:07.121115 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":132,"threads_started":1}
I20260808 02:13:07.123502 23769 heartbeater.cc:347] Connected to a master server at 127.18.184.190:44617
I20260808 02:13:07.123783 23769 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:07.124239 23769 heartbeater.cc:511] Master 127.18.184.190:44617 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
I20260808 02:13:07.125957 23564 ts_manager.cc:194] Registered new tserver with Master: f7bd2175ba344aff814ff98cd21e5804 (127.18.184.129:38803)
I20260808 02:13:07.127257 23564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:54135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:13:07.145471 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:44617
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:07.267730 23774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:07.268033 23774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:07.268082 23774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:07.272336 23774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:07.272419 23774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:07.272537 23774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:07.278389 23774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:44617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:07.279639 23774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:07.280766 23774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:07.290947 23781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:07.290975 23780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:07.291004 23774 server_base.cc:1034] running on GCE node
W20260808 02:13:07.290947 23783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:07.291626 23774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:07.292276 23774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:07.293442 23774 hybrid_clock.cc:648] HybridClock initialized: now 1786155187293421 us; error 42 us; skew 500 ppm
Aug 08 02:13:07 dist-test-slave-9gcw krb5kdc[23514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155187, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:07.296269 23774 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:13:07.296965 23774 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:07.297026 23774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:07.297258 23774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:07.299631 23774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "b9d0564cceb04de3a16586b75f68446f"
format_stamp: "Formatted at 2026-08-08 02:13:07 on dist-test-slave-9gcw"
server_key: "0c0b289674f44fa9a97b4fc64efef160"
server_key_iv: "6bd2f5d183fb727a81feefb5156ee764"
server_key_version: "encryptionkey@0"
I20260808 02:13:07.300192 23774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "b9d0564cceb04de3a16586b75f68446f"
format_stamp: "Formatted at 2026-08-08 02:13:07 on dist-test-slave-9gcw"
server_key: "0c0b289674f44fa9a97b4fc64efef160"
server_key_iv: "6bd2f5d183fb727a81feefb5156ee764"
server_key_version: "encryptionkey@0"
I20260808 02:13:07.304131 23774 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:13:07.306711 23789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:07.308115 23774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:13:07.308223 23774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "b9d0564cceb04de3a16586b75f68446f"
format_stamp: "Formatted at 2026-08-08 02:13:07 on dist-test-slave-9gcw"
server_key: "0c0b289674f44fa9a97b4fc64efef160"
server_key_iv: "6bd2f5d183fb727a81feefb5156ee764"
server_key_version: "encryptionkey@0"
I20260808 02:13:07.308322 23774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:07.340065 23774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:07.343619 23774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:07.343796 23774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:07.344434 23774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:07.345381 23774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:07.345432 23774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:07.345499 23774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:07.345535 23774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:07.354662 23774 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:40027
I20260808 02:13:07.354699 23911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:40027 every 8 connection(s)
I20260808 02:13:07.355283 23774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:07.364434 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 23774
I20260808 02:13:07.364578 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:07.364943 19170 external_mini_cluster.cc:1428] Setting key 262102bc5ede6583835165ec64d4db4a
Aug 08 02:13:07 dist-test-slave-9gcw krb5kdc[23514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155187, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:07.369885 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.357201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:33807 (local address 127.18.184.190:44617)
0808 02:13:07.357365 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:07.357370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:07.358347 (+ 977us) server_negotiation.cc:407] Connection header received
0808 02:13:07.359625 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:07.359628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:07.359695 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:07.359783 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:07.360979 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.361538 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.362661 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.362921 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.363008 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.365795 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:07.365799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:07.365801 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:07.365829 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:07.367322 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:07.367984 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:07.367988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:07.367989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:07.368083 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:07.368566 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:07.368569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:07.368571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:07.368857 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:07.368987 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:07.369568 (+ 581us) server_negotiation.cc:299] Negotiation successful
0808 02:13:07.369712 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:13:07.370843 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.357673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:44617 (local address 127.18.184.130:33807)
0808 02:13:07.358181 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:07.358225 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:07.359414 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:07.359977 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:07.359983 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:07.360360 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:07.360844 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.360858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.361649 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.361652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.362541 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.362550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.363074 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.363076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.363219 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.363797 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:07.363813 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:07.365598 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:07.367443 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:07.367452 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:07.367471 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:07.367853 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:07.368192 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:07.368199 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:07.368203 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:07.368437 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:07.368988 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:07.368996 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:07.369317 (+ 321us) client_negotiation.cc:769] Sending connection context
0808 02:13:07.369511 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 02:13:07.369865 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":144,"threads_started":1}
I20260808 02:13:07.372144 23912 heartbeater.cc:347] Connected to a master server at 127.18.184.190:44617
I20260808 02:13:07.372402 23912 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:07.372860 23912 heartbeater.cc:511] Master 127.18.184.190:44617 requested a full tablet report, sending...
I20260808 02:13:07.373963 23564 ts_manager.cc:194] Registered new tserver with Master: b9d0564cceb04de3a16586b75f68446f (127.18.184.130:40027)
I20260808 02:13:07.374576 23564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:33807
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:13:07.397130 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:44617
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:07.512399 23917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:07.512745 23917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:07.512796 23917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:07.516957 23917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:07.517027 23917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:07.517117 23917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:07.522723 23917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:44617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:07.523872 23917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:07.524964 23917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:07.534844 23922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:07.534940 23923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:07.534837 23925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:07.535051 23917 server_base.cc:1034] running on GCE node
I20260808 02:13:07.535549 23917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:07.536274 23917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:07.537497 23917 hybrid_clock.cc:648] HybridClock initialized: now 1786155187537463 us; error 49 us; skew 500 ppm
Aug 08 02:13:07 dist-test-slave-9gcw krb5kdc[23514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155187, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:07.540802 23917 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:13:07.541669 23917 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:07.541770 23917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:07.542021 23917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:07.544176 23917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "4498bf83765749389bf980b80e92aa55"
format_stamp: "Formatted at 2026-08-08 02:13:07 on dist-test-slave-9gcw"
server_key: "271baf2e781abbab7b62c538d7f96c14"
server_key_iv: "76329abbe4e1bde799d23772158eef9d"
server_key_version: "encryptionkey@0"
I20260808 02:13:07.544785 23917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "4498bf83765749389bf980b80e92aa55"
format_stamp: "Formatted at 2026-08-08 02:13:07 on dist-test-slave-9gcw"
server_key: "271baf2e781abbab7b62c538d7f96c14"
server_key_iv: "76329abbe4e1bde799d23772158eef9d"
server_key_version: "encryptionkey@0"
I20260808 02:13:07.548659 23917 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:13:07.551421 23932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:07.552862 23917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:07.553043 23917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "4498bf83765749389bf980b80e92aa55"
format_stamp: "Formatted at 2026-08-08 02:13:07 on dist-test-slave-9gcw"
server_key: "271baf2e781abbab7b62c538d7f96c14"
server_key_iv: "76329abbe4e1bde799d23772158eef9d"
server_key_version: "encryptionkey@0"
I20260808 02:13:07.553200 23917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:07.564749 23917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:07.567874 23917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:07.568094 23917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:07.568714 23917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:07.569713 23917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:07.569787 23917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:07.569885 23917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:07.569933 23917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:07.579167 23917 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:40415
I20260808 02:13:07.579192 24055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:40415 every 8 connection(s)
I20260808 02:13:07.579902 23917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:07.584985 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 23917
I20260808 02:13:07.585090 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:07.585527 19170 external_mini_cluster.cc:1428] Setting key 0d318504523091815148ef12fdd3463e
Aug 08 02:13:07 dist-test-slave-9gcw krb5kdc[23514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155187, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:07.594085 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.581951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:33673 (local address 127.18.184.190:44617)
0808 02:13:07.582130 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:07.582134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:07.582929 (+ 795us) server_negotiation.cc:407] Connection header received
0808 02:13:07.584323 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:07.584326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:07.584374 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:07.584457 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:07.585956 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.586605 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.587938 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.588162 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.588247 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.590772 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:07.590775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:07.590777 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:07.590801 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:07.592071 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:07.592550 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:07.592553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:07.592554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:07.592650 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:07.593052 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:07.593054 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:07.593055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:07.593291 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:07.593371 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:07.593859 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 02:13:07.593959 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:13:07.594945 24058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.582281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:44617 (local address 127.18.184.131:33673)
0808 02:13:07.582761 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:07.582810 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:07.584074 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:07.584645 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:07.584651 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:07.585129 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:07.585772 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.585789 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.586717 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.586719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.587804 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.587829 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.588294 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.588297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.588442 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.589081 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:07.589098 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:07.590605 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:07.592179 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:07.592184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:07.592195 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:07.592454 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:07.592778 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:07.592781 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:07.592783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:07.592957 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:07.593381 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:07.593386 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:07.593635 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 02:13:07.593798 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 02:13:07.594067 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":108,"threads_started":1}
I20260808 02:13:07.596171 24056 heartbeater.cc:347] Connected to a master server at 127.18.184.190:44617
I20260808 02:13:07.596415 24056 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:07.596868 24056 heartbeater.cc:511] Master 127.18.184.190:44617 requested a full tablet report, sending...
I20260808 02:13:07.597982 23564 ts_manager.cc:194] Registered new tserver with Master: 4498bf83765749389bf980b80e92aa55 (127.18.184.131:40415)
I20260808 02:13:07.598670 23564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:33673
I20260808 02:13:07.599681 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:13:07.609869 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.601877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54448 (local address 127.18.184.190:44617)
0808 02:13:07.601978 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:07.601982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:07.602247 (+ 265us) server_negotiation.cc:407] Connection header received
0808 02:13:07.602418 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:07.602421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:07.602471 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:07.602552 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:07.603260 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.603774 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.604648 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.604884 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.604993 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.606565 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:07.606569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:07.606572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:07.606596 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:07.608041 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:07.608481 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:07.608485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:07.608486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:07.608570 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:07.608889 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:07.608894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:07.608896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:07.609123 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:07.609324 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:07.609570 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 02:13:07.609687 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 02:13:07.612879 23564 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:07.616039 23564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:07.630626 24069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.625124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:38803 (local address 127.0.0.1:57036)
0808 02:13:07.626120 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:07.626135 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:07.626271 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:07.626775 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:07.626781 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:07.626803 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:07.626942 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.626949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.628166 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.628170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.629417 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.629427 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.630223 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.630232 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.630311 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.630342 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:13:07.630419 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 02:13:07.630503 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"mutex_wait_us":28,"thread_start_us":93,"threads_started":1}
I20260808 02:13:07.630687 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.625579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57036 (local address 127.18.184.129:38803)
0808 02:13:07.625971 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:07.625986 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:07.626238 (+ 252us) server_negotiation.cc:407] Connection header received
0808 02:13:07.626431 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:07.626435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:07.626608 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:07.626764 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:07.627066 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.627979 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.629541 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.630117 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.630250 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.630394 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:07.630446 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 02:13:07.630537 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":85,"threads_started":1}
I20260808 02:13:07.631467 24068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.624843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:40027 (local address 127.0.0.1:48832)
0808 02:13:07.625772 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:07.625836 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:07.625972 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:07.626946 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:07.626948 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:07.626973 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:07.627147 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.627154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.628638 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.628641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.630027 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.630033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.631075 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.631079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.631164 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.631187 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:13:07.631261 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:13:07.631330 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":151,"spinlock_wait_cycles":15616,"thread_start_us":95,"threads_started":1}
I20260808 02:13:07.632104 24072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.625286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48832 (local address 127.18.184.130:40027)
0808 02:13:07.626223 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:07.626240 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:07.626261 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:13:07.626373 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:07.626379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:07.626609 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:07.626764 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:07.627276 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.628489 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.630166 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.630950 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.631050 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.631127 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:07.631835 (+ 708us) server_negotiation.cc:299] Negotiation successful
0808 02:13:07.631930 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":125,"threads_started":1}
I20260808 02:13:07.632488 24067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.624660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:40415 (local address 127.0.0.1:44342)
0808 02:13:07.625774 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:07.625828 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:07.625971 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:07.626944 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:07.626946 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:07.626974 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:07.627118 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.627123 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.628638 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.628641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.630020 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.630033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.632110 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.632113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.632208 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.632230 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:13:07.632302 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 02:13:07.632368 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":923,"mutex_wait_us":105,"spinlock_wait_cycles":10880,"thread_start_us":95,"threads_started":1}
I20260808 02:13:07.632575 24070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.625208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44342 (local address 127.18.184.131:40415)
0808 02:13:07.626287 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:07.626301 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:07.626327 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:13:07.626462 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:07.626466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:07.626627 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:07.626789 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:07.627230 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.628468 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.631179 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.631980 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.632109 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.632240 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:07.632342 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 02:13:07.632429 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":963,"thread_start_us":172,"threads_started":1}
I20260808 02:13:07.634701 23982 tablet_service.cc:1467] Processing CreateTablet for tablet 22ab6a8e0d9049559b61b1ff28208bb9 (DEFAULT_TABLE table=test-table [id=4d1b96aa7348438e8faad65a9a5b9723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:07.634560 23695 tablet_service.cc:1467] Processing CreateTablet for tablet 22ab6a8e0d9049559b61b1ff28208bb9 (DEFAULT_TABLE table=test-table [id=4d1b96aa7348438e8faad65a9a5b9723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:07.634701 23840 tablet_service.cc:1467] Processing CreateTablet for tablet 22ab6a8e0d9049559b61b1ff28208bb9 (DEFAULT_TABLE table=test-table [id=4d1b96aa7348438e8faad65a9a5b9723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:07.635594 23982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22ab6a8e0d9049559b61b1ff28208bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:07.635655 23840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22ab6a8e0d9049559b61b1ff28208bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:07.635655 23695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22ab6a8e0d9049559b61b1ff28208bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:07.643110 24074 tablet_bootstrap.cc:492] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55: Bootstrap starting.
I20260808 02:13:07.643220 24075 tablet_bootstrap.cc:492] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804: Bootstrap starting.
I20260808 02:13:07.645464 24076 tablet_bootstrap.cc:492] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f: Bootstrap starting.
I20260808 02:13:07.645592 24075 tablet_bootstrap.cc:654] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:07.645591 24074 tablet_bootstrap.cc:654] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:07.646574 24075 log.cc:824] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:07.646855 24074 log.cc:824] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:07.647487 24076 tablet_bootstrap.cc:654] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:07.648556 24076 log.cc:824] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:07.648818 24074 tablet_bootstrap.cc:492] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55: No bootstrap required, opened a new log
I20260808 02:13:07.648833 24075 tablet_bootstrap.cc:492] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804: No bootstrap required, opened a new log
I20260808 02:13:07.649022 24074 ts_tablet_manager.cc:1397] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:13:07.649022 24075 ts_tablet_manager.cc:1397] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:13:07.653173 24074 raft_consensus.cc:348] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498bf83765749389bf980b80e92aa55" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 40415 } } peers { permanent_uuid: "b9d0564cceb04de3a16586b75f68446f" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40027 } } peers { permanent_uuid: "f7bd2175ba344aff814ff98cd21e5804" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 38803 } }
I20260808 02:13:07.653515 24074 raft_consensus.cc:374] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:07.653581 24074 raft_consensus.cc:729] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4498bf83765749389bf980b80e92aa55, State: Initialized, Role: FOLLOWER
I20260808 02:13:07.653519 24075 raft_consensus.cc:348] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498bf83765749389bf980b80e92aa55" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 40415 } } peers { permanent_uuid: "b9d0564cceb04de3a16586b75f68446f" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40027 } } peers { permanent_uuid: "f7bd2175ba344aff814ff98cd21e5804" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 38803 } }
I20260808 02:13:07.653761 24075 raft_consensus.cc:374] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:07.653844 24075 raft_consensus.cc:729] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7bd2175ba344aff814ff98cd21e5804, State: Initialized, Role: FOLLOWER
I20260808 02:13:07.654101 24074 consensus_queue.cc:260] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498bf83765749389bf980b80e92aa55" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 40415 } } peers { permanent_uuid: "b9d0564cceb04de3a16586b75f68446f" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40027 } } peers { permanent_uuid: "f7bd2175ba344aff814ff98cd21e5804" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 38803 } }
I20260808 02:13:07.654228 24075 consensus_queue.cc:260] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498bf83765749389bf980b80e92aa55" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 40415 } } peers { permanent_uuid: "b9d0564cceb04de3a16586b75f68446f" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40027 } } peers { permanent_uuid: "f7bd2175ba344aff814ff98cd21e5804" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 38803 } }
I20260808 02:13:07.655156 24074 ts_tablet_manager.cc:1428] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:13:07.655156 24075 ts_tablet_manager.cc:1428] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:13:07.655565 24056 heartbeater.cc:503] Master 127.18.184.190:44617 was elected leader, sending a full tablet report...
I20260808 02:13:07.655689 24076 tablet_bootstrap.cc:492] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f: No bootstrap required, opened a new log
I20260808 02:13:07.655887 24076 ts_tablet_manager.cc:1397] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260808 02:13:07.656155 23769 heartbeater.cc:503] Master 127.18.184.190:44617 was elected leader, sending a full tablet report...
I20260808 02:13:07.659646 24076 raft_consensus.cc:348] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498bf83765749389bf980b80e92aa55" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 40415 } } peers { permanent_uuid: "b9d0564cceb04de3a16586b75f68446f" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40027 } } peers { permanent_uuid: "f7bd2175ba344aff814ff98cd21e5804" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 38803 } }
I20260808 02:13:07.659945 24076 raft_consensus.cc:374] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:07.660014 24076 raft_consensus.cc:729] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9d0564cceb04de3a16586b75f68446f, State: Initialized, Role: FOLLOWER
I20260808 02:13:07.660423 24076 consensus_queue.cc:260] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498bf83765749389bf980b80e92aa55" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 40415 } } peers { permanent_uuid: "b9d0564cceb04de3a16586b75f68446f" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40027 } } peers { permanent_uuid: "f7bd2175ba344aff814ff98cd21e5804" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 38803 } }
I20260808 02:13:07.661031 23912 heartbeater.cc:503] Master 127.18.184.190:44617 was elected leader, sending a full tablet report...
I20260808 02:13:07.661290 24076 ts_tablet_manager.cc:1428] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 02:13:07.791752 24081 raft_consensus.cc:482] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:07.792008 24081 raft_consensus.cc:504] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498bf83765749389bf980b80e92aa55" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 40415 } } peers { permanent_uuid: "b9d0564cceb04de3a16586b75f68446f" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40027 } } peers { permanent_uuid: "f7bd2175ba344aff814ff98cd21e5804" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 38803 } }
I20260808 02:13:07.793292 24081 leader_election.cc:288] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9d0564cceb04de3a16586b75f68446f (127.18.184.130:40027), f7bd2175ba344aff814ff98cd21e5804 (127.18.184.129:38803)
I20260808 02:13:07.797565 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.793703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:43027 (local address 127.18.184.129:38803)
0808 02:13:07.793875 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:07.793881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:07.794185 (+ 304us) server_negotiation.cc:407] Connection header received
0808 02:13:07.794316 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:07.794319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:07.794367 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:07.794450 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:07.794758 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.795248 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.796452 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.796876 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.797008 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.797071 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:07.797356 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 02:13:07.797427 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 02:13:07.797570 24058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.793598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:40027 (local address 127.18.184.131:57757)
0808 02:13:07.793830 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:07.793848 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:07.793956 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:07.794313 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:07.794316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:07.794336 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:07.794477 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.794485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.795435 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.795438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.796325 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.796335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.797006 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.797010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.797234 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.797259 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:13:07.797354 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 02:13:07.797427 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":139}
I20260808 02:13:07.797574 24072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.793899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:57757 (local address 127.18.184.130:40027)
0808 02:13:07.794034 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:07.794039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:07.794056 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:13:07.794119 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:07.794125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:07.794183 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:07.794264 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:07.794603 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.795266 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.796488 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.796878 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.797008 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.797070 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:07.797383 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 02:13:07.797452 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:13:07.797590 24085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.793598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:38803 (local address 127.18.184.131:43027)
0808 02:13:07.794080 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:07.794095 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:07.794197 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:07.794469 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:07.794472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:07.794486 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:07.794629 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.794637 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.795383 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.795390 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.796311 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:07.796322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.797006 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.797010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:07.797229 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.797251 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:13:07.797312 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 02:13:07.797401 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":68,"thread_start_us":157,"threads_started":1}
I20260808 02:13:07.798372 23861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22ab6a8e0d9049559b61b1ff28208bb9" candidate_uuid: "4498bf83765749389bf980b80e92aa55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9d0564cceb04de3a16586b75f68446f" is_pre_election: true
I20260808 02:13:07.798416 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22ab6a8e0d9049559b61b1ff28208bb9" candidate_uuid: "4498bf83765749389bf980b80e92aa55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7bd2175ba344aff814ff98cd21e5804" is_pre_election: true
I20260808 02:13:07.798681 23861 raft_consensus.cc:2454] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4498bf83765749389bf980b80e92aa55 in term 0.
I20260808 02:13:07.798688 23719 raft_consensus.cc:2454] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4498bf83765749389bf980b80e92aa55 in term 0.
I20260808 02:13:07.799157 23935 leader_election.cc:302] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4498bf83765749389bf980b80e92aa55, b9d0564cceb04de3a16586b75f68446f; no voters:
I20260808 02:13:07.799510 24081 raft_consensus.cc:2781] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:07.799607 24081 raft_consensus.cc:482] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:07.799666 24081 raft_consensus.cc:3037] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:07.800802 24081 raft_consensus.cc:504] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498bf83765749389bf980b80e92aa55" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 40415 } } peers { permanent_uuid: "b9d0564cceb04de3a16586b75f68446f" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40027 } } peers { permanent_uuid: "f7bd2175ba344aff814ff98cd21e5804" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 38803 } }
I20260808 02:13:07.801198 24081 leader_election.cc:288] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [CANDIDATE]: Term 1 election: Requested vote from peers b9d0564cceb04de3a16586b75f68446f (127.18.184.130:40027), f7bd2175ba344aff814ff98cd21e5804 (127.18.184.129:38803)
I20260808 02:13:07.801519 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22ab6a8e0d9049559b61b1ff28208bb9" candidate_uuid: "4498bf83765749389bf980b80e92aa55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7bd2175ba344aff814ff98cd21e5804"
I20260808 02:13:07.801539 23861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22ab6a8e0d9049559b61b1ff28208bb9" candidate_uuid: "4498bf83765749389bf980b80e92aa55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9d0564cceb04de3a16586b75f68446f"
I20260808 02:13:07.801631 23719 raft_consensus.cc:3037] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:07.801648 23861 raft_consensus.cc:3037] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:07.802611 23719 raft_consensus.cc:2454] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4498bf83765749389bf980b80e92aa55 in term 1.
I20260808 02:13:07.802896 23934 leader_election.cc:302] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4498bf83765749389bf980b80e92aa55, f7bd2175ba344aff814ff98cd21e5804; no voters:
I20260808 02:13:07.802929 23861 raft_consensus.cc:2454] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4498bf83765749389bf980b80e92aa55 in term 1.
I20260808 02:13:07.803061 24081 raft_consensus.cc:2781] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:07.803294 24081 raft_consensus.cc:686] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 1 LEADER]: Becoming Leader. State: Replica: 4498bf83765749389bf980b80e92aa55, State: Running, Role: LEADER
I20260808 02:13:07.803602 24081 consensus_queue.cc:237] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498bf83765749389bf980b80e92aa55" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 40415 } } peers { permanent_uuid: "b9d0564cceb04de3a16586b75f68446f" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40027 } } peers { permanent_uuid: "f7bd2175ba344aff814ff98cd21e5804" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 38803 } }
I20260808 02:13:07.809674 23564 catalog_manager.cc:5495] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4498bf83765749389bf980b80e92aa55 (127.18.184.131). New cstate: current_term: 1 leader_uuid: "4498bf83765749389bf980b80e92aa55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498bf83765749389bf980b80e92aa55" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9d0564cceb04de3a16586b75f68446f" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7bd2175ba344aff814ff98cd21e5804" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 38803 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:07.820942 24070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.817368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44346 (local address 127.18.184.131:40415)
0808 02:13:07.817510 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:07.817515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:07.817531 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:13:07.817639 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:07.817642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:07.817744 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:07.817847 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:07.818139 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.818641 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.819526 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.819749 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.819831 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.820094 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:07.820564 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:07.820657 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:07.820717 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 02:13:07.820769 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:13:07.827219 23861 raft_consensus.cc:1261] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f [term 1 FOLLOWER]: Refusing update from remote peer 4498bf83765749389bf980b80e92aa55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:07.827219 23719 raft_consensus.cc:1261] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804 [term 1 FOLLOWER]: Refusing update from remote peer 4498bf83765749389bf980b80e92aa55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:07.828001 24086 consensus_queue.cc:1035] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7bd2175ba344aff814ff98cd21e5804" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 38803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:07.828208 24081 consensus_queue.cc:1035] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9d0564cceb04de3a16586b75f68446f" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 40027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:13:07.830797 24057 tablet.cc:2369] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:07.834301 24089 mvcc.cc:205] Tried to move back new op lower bound from 7316091649332621312 to 7316091649246326784. Current Snapshot: MvccSnapshot[applied={T|T < 7316091649332621312}]
I20260808 02:13:07.836344 24091 mvcc.cc:205] Tried to move back new op lower bound from 7316091649332621312 to 7316091649246326784. Current Snapshot: MvccSnapshot[applied={T|T < 7316091649332621312}]
I20260808 02:13:07.837258 24090 mvcc.cc:205] Tried to move back new op lower bound from 7316091649332621312 to 7316091649246326784. Current Snapshot: MvccSnapshot[applied={T|T < 7316091649332621312}]
I20260808 02:13:07.843166 23564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:07.843351 23564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:07.845672 23564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e01d57b362c246519b53828a13a12398: Sending DeleteTablet for 3 replicas of tablet 22ab6a8e0d9049559b61b1ff28208bb9
I20260808 02:13:07.846474 23982 tablet_service.cc:1514] Processing DeleteTablet for tablet 22ab6a8e0d9049559b61b1ff28208bb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:07 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:44342
I20260808 02:13:07.846474 23840 tablet_service.cc:1514] Processing DeleteTablet for tablet 22ab6a8e0d9049559b61b1ff28208bb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:07 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:48832
I20260808 02:13:07.846546 23695 tablet_service.cc:1514] Processing DeleteTablet for tablet 22ab6a8e0d9049559b61b1ff28208bb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:07 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:57036
I20260808 02:13:07.847086 24101 tablet_replica.cc:330] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804: stopping tablet replica
I20260808 02:13:07.847201 24099 tablet_replica.cc:330] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55: stopping tablet replica
I20260808 02:13:07.847432 24101 raft_consensus.cc:2229] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:07.847497 24099 raft_consensus.cc:2229] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:07.847777 24101 raft_consensus.cc:2258] T 22ab6a8e0d9049559b61b1ff28208bb9 P f7bd2175ba344aff814ff98cd21e5804 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:07.847853 24099 raft_consensus.cc:2258] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:07.848241 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 23632
I20260808 02:13:07.851184 24100 tablet_replica.cc:330] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f: stopping tablet replica
I20260808 02:13:07.851393 24099 ts_tablet_manager.cc:1905] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:07.851539 24100 raft_consensus.cc:2229] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:07.851922 24100 raft_consensus.cc:2258] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:07.853533 24100 ts_tablet_manager.cc:1905] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:07.855118 24099 ts_tablet_manager.cc:1918] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:07.855221 24099 log.cc:1196] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/22ab6a8e0d9049559b61b1ff28208bb9
I20260808 02:13:07.855600 24099 ts_tablet_manager.cc:1939] T 22ab6a8e0d9049559b61b1ff28208bb9 P 4498bf83765749389bf980b80e92aa55: Deleting consensus metadata
I20260808 02:13:07.856887 23547 catalog_manager.cc:4841] TS 4498bf83765749389bf980b80e92aa55 (127.18.184.131:40415): tablet 22ab6a8e0d9049559b61b1ff28208bb9 (table test-table [id=4d1b96aa7348438e8faad65a9a5b9723]) successfully deleted
I20260808 02:13:07.857360 24100 ts_tablet_manager.cc:1918] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:07.857470 24100 log.cc:1196] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/22ab6a8e0d9049559b61b1ff28208bb9
I20260808 02:13:07.857795 24100 ts_tablet_manager.cc:1939] T 22ab6a8e0d9049559b61b1ff28208bb9 P b9d0564cceb04de3a16586b75f68446f: Deleting consensus metadata
W20260808 02:13:07.858561 23549 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:38803 (error 108)
I20260808 02:13:07.859119 23550 catalog_manager.cc:4841] TS b9d0564cceb04de3a16586b75f68446f (127.18.184.130:40027): tablet 22ab6a8e0d9049559b61b1ff28208bb9 (table test-table [id=4d1b96aa7348438e8faad65a9a5b9723]) successfully deleted
I20260808 02:13:07.859673 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 23774
I20260808 02:13:07.859716 24067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.859406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:38803 (local address 127.0.0.1:57042)
0808 02:13:07.859550 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:07.859627 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:38803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:13:07.859961 23549 catalog_manager.cc:4584] TS f7bd2175ba344aff814ff98cd21e5804 (127.18.184.129:38803): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 22ab6a8e0d9049559b61b1ff28208bb9: Network error: Client connection negotiation failed: client connection to 127.18.184.129:38803: connect: Connection refused (error 111)
I20260808 02:13:07.867727 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 23917
I20260808 02:13:07.876406 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 23529
2026-08-08T02:13:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2580 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T02:13:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:07Z Disabled control of system clock
I20260808 02:13:07.911327 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:36875
--webserver_interface=127.18.184.190
--webserver_port=0
--builtin_ntp_servers=127.18.184.148:34357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:36875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:13:08.025013 24107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:08.025381 24107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:08.025424 24107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:08.029490 24107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:08.029556 24107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:08.029572 24107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:08.029589 24107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:08.029604 24107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:08.034416 24107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:36875
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:36875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:08.035557 24107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:08.037015 24107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:08.045964 24115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:08.046093 24114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:08.046093 24117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:08.046232 24107 server_base.cc:1034] running on GCE node
I20260808 02:13:08.046944 24107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:08.048096 24107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:08.049224 24107 hybrid_clock.cc:648] HybridClock initialized: now 1786155188049181 us; error 64 us; skew 500 ppm
I20260808 02:13:08.051185 24107 webserver.cc:460] Webserver started at http://127.18.184.190:35877/ using document root <none> and password file <none>
I20260808 02:13:08.051785 24107 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:08.051852 24107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:08.052098 24107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:08.054417 24107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "9fa7e042bf2d4e8f91db6bafda142c7f"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-9gcw"
server_key: "60c9f4b60d971978a5de19f38b82dc4f"
server_key_iv: "8662d94fb320c1706ca22933ee9cfe5e"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.055016 24107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "9fa7e042bf2d4e8f91db6bafda142c7f"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-9gcw"
server_key: "60c9f4b60d971978a5de19f38b82dc4f"
server_key_iv: "8662d94fb320c1706ca22933ee9cfe5e"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.058917 24107 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:13:08.061555 24123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.062958 24107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:13:08.063108 24107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "9fa7e042bf2d4e8f91db6bafda142c7f"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-9gcw"
server_key: "60c9f4b60d971978a5de19f38b82dc4f"
server_key_iv: "8662d94fb320c1706ca22933ee9cfe5e"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.063241 24107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:08.067996 24107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:08.068611 24107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:08.068796 24107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:08.076103 24107 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:36875
I20260808 02:13:08.076112 24180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:36875 every 8 connection(s)
I20260808 02:13:08.077060 24107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:08.077363 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 24107
I20260808 02:13:08.077476 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:08.077875 19170 external_mini_cluster.cc:1428] Setting key 4ae3de9c27bd33528ff433d9a1a8f665
I20260808 02:13:08.080295 24181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:08.084566 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.079356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40050 (local address 127.18.184.190:36875)
0808 02:13:08.080500 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.080521 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.080566 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:13:08.081492 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.081515 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.081953 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.082334 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.083048 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.083053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.083067 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:08.083077 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.083139 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:08.083161 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.083318 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.083386 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.083641 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":539,"threads_started":1}
I20260808 02:13:08.087255 24181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f: Bootstrap starting.
I20260808 02:13:08.089502 24181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:08.090318 24181 log.cc:824] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:08.092279 24181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f: No bootstrap required, opened a new log
I20260808 02:13:08.096009 24181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa7e042bf2d4e8f91db6bafda142c7f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36875 } }
I20260808 02:13:08.096331 24181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:08.096431 24181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fa7e042bf2d4e8f91db6bafda142c7f, State: Initialized, Role: FOLLOWER
I20260808 02:13:08.096915 24181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa7e042bf2d4e8f91db6bafda142c7f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36875 } }
I20260808 02:13:08.097074 24181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:08.097177 24181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:08.097285 24181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:08.098206 24181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa7e042bf2d4e8f91db6bafda142c7f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36875 } }
I20260808 02:13:08.098542 24181 leader_election.cc:302] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fa7e042bf2d4e8f91db6bafda142c7f; no voters:
I20260808 02:13:08.098873 24181 leader_election.cc:288] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:08.099023 24186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:08.099279 24186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [term 1 LEADER]: Becoming Leader. State: Replica: 9fa7e042bf2d4e8f91db6bafda142c7f, State: Running, Role: LEADER
I20260808 02:13:08.099598 24186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa7e042bf2d4e8f91db6bafda142c7f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36875 } }
I20260808 02:13:08.100144 24181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:08.101320 24187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fa7e042bf2d4e8f91db6bafda142c7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa7e042bf2d4e8f91db6bafda142c7f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36875 } } }
I20260808 02:13:08.101449 24187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:08.101673 24188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fa7e042bf2d4e8f91db6bafda142c7f. Latest consensus state: current_term: 1 leader_uuid: "9fa7e042bf2d4e8f91db6bafda142c7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa7e042bf2d4e8f91db6bafda142c7f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36875 } } }
I20260808 02:13:08.101778 24188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:08.102072 24197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:08.104745 24197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:08.110669 24197 catalog_manager.cc:1347] Generated new cluster ID: 7fe1f2e5583e439ebadaea42e16f4a71
I20260808 02:13:08.110759 24197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:08.122927 24197 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:08.123347 24197 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:08.138223 24197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f: Generated new TSK 0
I20260808 02:13:08.139083 24197 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:08.143414 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36875
--builtin_ntp_servers=127.18.184.148:34357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:13:08.262758 24208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:08.263108 24208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:08.263201 24208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:08.267694 24208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:08.267796 24208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:08.267892 24208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:08.273002 24208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:08.274266 24208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:08.275810 24208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:08.286041 24216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:08.285986 24214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:08.286290 24208 server_base.cc:1034] running on GCE node
W20260808 02:13:08.286306 24213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:08.286744 24208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:08.287410 24208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:08.288605 24208 hybrid_clock.cc:648] HybridClock initialized: now 1786155188288573 us; error 43 us; skew 500 ppm
I20260808 02:13:08.290584 24208 webserver.cc:460] Webserver started at http://127.18.184.129:36791/ using document root <none> and password file <none>
I20260808 02:13:08.291198 24208 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:08.291291 24208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:08.291543 24208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:08.293744 24208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "969ed5962f33469b97680e8e535fa086"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-9gcw"
server_key: "baf1b06fba51379c793e614a9114117e"
server_key_iv: "d5e5e3833815f06bd85b5941de4646ec"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.294356 24208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "969ed5962f33469b97680e8e535fa086"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-9gcw"
server_key: "baf1b06fba51379c793e614a9114117e"
server_key_iv: "d5e5e3833815f06bd85b5941de4646ec"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.298086 24208 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:13:08.300922 24222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.302461 24208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:13:08.302567 24208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "969ed5962f33469b97680e8e535fa086"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-9gcw"
server_key: "baf1b06fba51379c793e614a9114117e"
server_key_iv: "d5e5e3833815f06bd85b5941de4646ec"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.302670 24208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:08.307350 24208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:08.307945 24208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:08.308089 24208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:08.308604 24208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:08.309662 24208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:08.309713 24208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.309753 24208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:08.309769 24208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.318758 24347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:43199 every 8 connection(s)
I20260808 02:13:08.318819 24208 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:43199
I20260808 02:13:08.319752 24208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:08.321256 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 24208
I20260808 02:13:08.321401 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:08.321903 19170 external_mini_cluster.cc:1428] Setting key 90db9a45907b1db653144b60bb3e3b54
I20260808 02:13:08.324187 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36875
--builtin_ntp_servers=127.18.184.148:34357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 02:13:08.327201 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.321940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:40065 (local address 127.18.184.190:36875)
0808 02:13:08.322134 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.322139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.323263 (+ 1124us) server_negotiation.cc:407] Connection header received
0808 02:13:08.325065 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.325068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.325116 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.325247 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.326706 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.326709 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.326711 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:08.326724 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.326747 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:08.326755 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.326855 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.327037 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.327079 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 02:13:08.328616 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.322357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:36875 (local address 127.18.184.129:40065)
0808 02:13:08.323093 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:08.323140 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:08.324822 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:08.325441 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:08.325449 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:08.325561 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:08.326306 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:08.326309 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:08.326383 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:08.326499 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:08.326861 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:08.326874 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:08.326883 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 02:13:08.327096 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 02:13:08.327433 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":119,"threads_started":1}
I20260808 02:13:08.330147 24348 heartbeater.cc:347] Connected to a master server at 127.18.184.190:36875
I20260808 02:13:08.330464 24348 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:08.331087 24348 heartbeater.cc:511] Master 127.18.184.190:36875 requested a full tablet report, sending...
I20260808 02:13:08.333083 24144 ts_manager.cc:194] Registered new tserver with Master: 969ed5962f33469b97680e8e535fa086 (127.18.184.129:43199)
W20260808 02:13:08.455714 24351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:08.455996 24351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:08.456079 24351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:08.460326 24351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:08.460383 24351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:08.460485 24351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:08.465342 24351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:08.466506 24351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:08.468102 24351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:08.478765 24356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:08.478716 24359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:08.478689 24357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:08.479053 24351 server_base.cc:1034] running on GCE node
I20260808 02:13:08.479578 24351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:08.480418 24351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:08.481611 24351 hybrid_clock.cc:648] HybridClock initialized: now 1786155188481594 us; error 28 us; skew 500 ppm
I20260808 02:13:08.483728 24351 webserver.cc:460] Webserver started at http://127.18.184.130:42207/ using document root <none> and password file <none>
I20260808 02:13:08.484316 24351 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:08.484380 24351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:08.484558 24351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:08.486815 24351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "ddd5881f97d2462a8f50498e0613c2d6"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-9gcw"
server_key: "ad6f52d0ed00a290bfefb22d51f51c40"
server_key_iv: "dbd9ba244655095fad905a3db92b55bb"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.487345 24351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "ddd5881f97d2462a8f50498e0613c2d6"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-9gcw"
server_key: "ad6f52d0ed00a290bfefb22d51f51c40"
server_key_iv: "dbd9ba244655095fad905a3db92b55bb"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.491289 24351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:13:08.493942 24365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.495291 24351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 02:13:08.495458 24351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "ddd5881f97d2462a8f50498e0613c2d6"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-9gcw"
server_key: "ad6f52d0ed00a290bfefb22d51f51c40"
server_key_iv: "dbd9ba244655095fad905a3db92b55bb"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.495582 24351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:08.500511 24351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:08.501289 24351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:08.501482 24351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:08.502059 24351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:08.503073 24351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:08.503121 24351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.503160 24351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:08.503185 24351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.512198 24351 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:45189
I20260808 02:13:08.512228 24486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:45189 every 8 connection(s)
I20260808 02:13:08.513125 24351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:08.519586 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.515007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:39393 (local address 127.18.184.190:36875)
0808 02:13:08.515161 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.515165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.516293 (+ 1128us) server_negotiation.cc:407] Connection header received
0808 02:13:08.517715 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.517718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.517776 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.517886 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.518919 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.518923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.518925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:08.518938 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.518967 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:08.518975 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.519083 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.519398 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.519446 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:13:08.520627 24490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.515441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:36875 (local address 127.18.184.130:39393)
0808 02:13:08.516119 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:08.516161 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:08.517507 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:08.518050 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:08.518056 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:08.518141 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:08.518659 (+ 518us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:08.518661 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:08.518701 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:08.518765 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:08.519180 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:08.519189 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:08.519214 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:13:08.519440 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 02:13:08.519682 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":112,"threads_started":1}
I20260808 02:13:08.521828 24487 heartbeater.cc:347] Connected to a master server at 127.18.184.190:36875
I20260808 02:13:08.522074 24487 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:08.522497 24487 heartbeater.cc:511] Master 127.18.184.190:36875 requested a full tablet report, sending...
I20260808 02:13:08.523399 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 24351
I20260808 02:13:08.523531 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:08.523597 24144 ts_manager.cc:194] Registered new tserver with Master: ddd5881f97d2462a8f50498e0613c2d6 (127.18.184.130:45189)
I20260808 02:13:08.523855 19170 external_mini_cluster.cc:1428] Setting key 874578fac72a88ba95c598077bdf366a
I20260808 02:13:08.525823 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36875
--builtin_ntp_servers=127.18.184.148:34357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:13:08.641666 24491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:08.641969 24491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:08.642010 24491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:08.646469 24491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:08.646538 24491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:08.646596 24491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:08.651539 24491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:08.652823 24491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:08.654353 24491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:08.664464 24497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:08.664459 24501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:08.664512 24491 server_base.cc:1034] running on GCE node
W20260808 02:13:08.664458 24498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:08.665131 24491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:08.665936 24491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:08.667181 24491 hybrid_clock.cc:648] HybridClock initialized: now 1786155188667131 us; error 56 us; skew 500 ppm
I20260808 02:13:08.669649 24491 webserver.cc:460] Webserver started at http://127.18.184.131:32779/ using document root <none> and password file <none>
I20260808 02:13:08.670295 24491 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:08.670397 24491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:08.670624 24491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:08.672894 24491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "a1fca9b7d7994fc0ac07b04f4501ea23"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-9gcw"
server_key: "465709fd21c90a81ebca7a2bd06732df"
server_key_iv: "11cd2647ef37a9b49e271bd37f8e903d"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.673548 24491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "a1fca9b7d7994fc0ac07b04f4501ea23"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-9gcw"
server_key: "465709fd21c90a81ebca7a2bd06732df"
server_key_iv: "11cd2647ef37a9b49e271bd37f8e903d"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.677443 24491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:13:08.680136 24507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.681540 24491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:08.681680 24491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "a1fca9b7d7994fc0ac07b04f4501ea23"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-9gcw"
server_key: "465709fd21c90a81ebca7a2bd06732df"
server_key_iv: "11cd2647ef37a9b49e271bd37f8e903d"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.681813 24491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:08.686741 24491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:08.687358 24491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:08.687549 24491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:08.688112 24491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:08.689044 24491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:08.689118 24491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.689224 24491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:08.689271 24491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.698645 24491 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:42463
I20260808 02:13:08.698683 24625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:42463 every 8 connection(s)
I20260808 02:13:08.699585 24491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:08.703310 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 24491
I20260808 02:13:08.703437 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:08.703703 19170 external_mini_cluster.cc:1428] Setting key 6c7d23d70be320abc1e05001fa4d18f5
I20260808 02:13:08.706564 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.701606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:58575 (local address 127.18.184.190:36875)
0808 02:13:08.701790 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.701795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.702936 (+ 1141us) server_negotiation.cc:407] Connection header received
0808 02:13:08.704399 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.704402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.704460 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.704551 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.706032 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.706035 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.706038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:08.706051 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.706081 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:08.706091 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.706220 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.706418 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.706460 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 02:13:08.707540 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.702065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:36875 (local address 127.18.184.131:58575)
0808 02:13:08.702782 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:08.702826 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:08.704148 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:08.704804 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:08.704811 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:08.704904 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:08.705704 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:08.705706 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:08.705747 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:08.705871 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:08.706273 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:08.706281 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:08.706285 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:08.706424 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 02:13:08.706655 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":129,"threads_started":1}
I20260808 02:13:08.708647 24626 heartbeater.cc:347] Connected to a master server at 127.18.184.190:36875
I20260808 02:13:08.708885 24626 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:08.709362 24626 heartbeater.cc:511] Master 127.18.184.190:36875 requested a full tablet report, sending...
I20260808 02:13:08.710412 24144 ts_manager.cc:194] Registered new tserver with Master: a1fca9b7d7994fc0ac07b04f4501ea23 (127.18.184.131:42463)
I20260808 02:13:08.718043 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:13:08.721760 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.720133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40060 (local address 127.18.184.190:36875)
0808 02:13:08.720321 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.720326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.720430 (+ 104us) server_negotiation.cc:407] Connection header received
0808 02:13:08.720532 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.720534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.720577 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.720646 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.721377 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.721379 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.721381 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:08.721392 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.721411 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:08.721419 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.721490 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.721556 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.721588 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:13:08.724594 24144 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:08.727962 24144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:08.739885 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.736760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:45189 (local address 127.0.0.1:39056)
0808 02:13:08.737750 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:08.737763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:08.737893 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:08.738756 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:08.738760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:08.738776 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:08.739304 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:08.739305 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:08.739315 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:08.739337 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:08.739689 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:08.739692 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:08.739694 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:13:08.739762 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:13:08.739787 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"thread_start_us":103,"threads_started":1}
I20260808 02:13:08.740193 24642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.736836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49136 (local address 127.18.184.131:42463)
0808 02:13:08.737339 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.737349 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.737770 (+ 421us) server_negotiation.cc:407] Connection header received
0808 02:13:08.738021 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.738027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.738238 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.738398 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.739266 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.739274 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.739287 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:08.739299 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.739350 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:08.739384 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.739489 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.739959 (+ 470us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.740046 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":159,"threads_started":1}
I20260808 02:13:08.740481 24644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.737567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39056 (local address 127.18.184.130:45189)
0808 02:13:08.738012 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.738026 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.738050 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:13:08.738105 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.738110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.738309 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.738447 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.739472 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.739479 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.739492 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:08.739506 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.739557 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:08.739595 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.739702 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.740086 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.740143 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":112,"threads_started":1}
I20260808 02:13:08.739884 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.736672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:42463 (local address 127.0.0.1:49136)
0808 02:13:08.737624 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:08.737670 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:08.737871 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:08.738394 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:08.738399 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:08.738469 (+ 70us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:08.739064 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:08.739066 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:08.739087 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:08.739156 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:08.739534 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:08.739543 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:08.739549 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:13:08.739653 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 02:13:08.739725 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"thread_start_us":84,"threads_started":1}
I20260808 02:13:08.740973 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.737454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:43199 (local address 127.0.0.1:54490)
0808 02:13:08.738089 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:08.738101 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:08.738159 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:08.739119 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:08.739124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:08.739140 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:08.739776 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:08.739777 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:08.739788 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:08.739814 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:08.740712 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:08.740715 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:08.740719 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:08.740823 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 02:13:08.740859 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":92,"threads_started":1}
I20260808 02:13:08.741273 24645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.737639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54490 (local address 127.18.184.129:43199)
0808 02:13:08.738149 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.738168 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.738197 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:13:08.738244 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.738248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.738382 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.738494 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.740452 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.740457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.740467 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:08.740477 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.740516 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:08.740543 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.740664 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.740859 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.740937 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":164,"threads_started":1}
I20260808 02:13:08.743510 24558 tablet_service.cc:1467] Processing CreateTablet for tablet 06301ab250b84b099b3e9a639e9f232e (DEFAULT_TABLE table=test-table [id=592afc4b1def4a4fa3544b3e693429c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:08.743455 24418 tablet_service.cc:1467] Processing CreateTablet for tablet 06301ab250b84b099b3e9a639e9f232e (DEFAULT_TABLE table=test-table [id=592afc4b1def4a4fa3544b3e693429c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:08.743726 24274 tablet_service.cc:1467] Processing CreateTablet for tablet 06301ab250b84b099b3e9a639e9f232e (DEFAULT_TABLE table=test-table [id=592afc4b1def4a4fa3544b3e693429c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:08.744547 24418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06301ab250b84b099b3e9a639e9f232e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:08.744581 24558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06301ab250b84b099b3e9a639e9f232e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:08.744581 24274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06301ab250b84b099b3e9a639e9f232e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:08.751179 24646 tablet_bootstrap.cc:492] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6: Bootstrap starting.
I20260808 02:13:08.752447 24648 tablet_bootstrap.cc:492] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23: Bootstrap starting.
I20260808 02:13:08.752447 24647 tablet_bootstrap.cc:492] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086: Bootstrap starting.
I20260808 02:13:08.754155 24646 tablet_bootstrap.cc:654] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:08.754644 24647 tablet_bootstrap.cc:654] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:08.754644 24648 tablet_bootstrap.cc:654] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:08.755009 24646 log.cc:824] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:08.755528 24647 log.cc:824] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:08.755528 24648 log.cc:824] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:08.757334 24646 tablet_bootstrap.cc:492] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6: No bootstrap required, opened a new log
I20260808 02:13:08.757426 24648 tablet_bootstrap.cc:492] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23: No bootstrap required, opened a new log
I20260808 02:13:08.757552 24646 ts_tablet_manager.cc:1397] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:13:08.757616 24648 ts_tablet_manager.cc:1397] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 02:13:08.757699 24647 tablet_bootstrap.cc:492] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086: No bootstrap required, opened a new log
I20260808 02:13:08.757874 24647 ts_tablet_manager.cc:1397] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:13:08.760994 24648 raft_consensus.cc:348] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42463 } } peers { permanent_uuid: "ddd5881f97d2462a8f50498e0613c2d6" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 45189 } } peers { permanent_uuid: "969ed5962f33469b97680e8e535fa086" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 43199 } }
I20260808 02:13:08.761296 24648 raft_consensus.cc:374] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:08.761368 24648 raft_consensus.cc:729] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1fca9b7d7994fc0ac07b04f4501ea23, State: Initialized, Role: FOLLOWER
I20260808 02:13:08.761935 24648 consensus_queue.cc:260] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42463 } } peers { permanent_uuid: "ddd5881f97d2462a8f50498e0613c2d6" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 45189 } } peers { permanent_uuid: "969ed5962f33469b97680e8e535fa086" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 43199 } }
I20260808 02:13:08.762369 24646 raft_consensus.cc:348] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42463 } } peers { permanent_uuid: "ddd5881f97d2462a8f50498e0613c2d6" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 45189 } } peers { permanent_uuid: "969ed5962f33469b97680e8e535fa086" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 43199 } }
I20260808 02:13:08.762369 24647 raft_consensus.cc:348] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42463 } } peers { permanent_uuid: "ddd5881f97d2462a8f50498e0613c2d6" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 45189 } } peers { permanent_uuid: "969ed5962f33469b97680e8e535fa086" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 43199 } }
I20260808 02:13:08.762648 24646 raft_consensus.cc:374] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:08.762648 24647 raft_consensus.cc:374] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:08.762708 24646 raft_consensus.cc:729] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd5881f97d2462a8f50498e0613c2d6, State: Initialized, Role: FOLLOWER
I20260808 02:13:08.762708 24647 raft_consensus.cc:729] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 969ed5962f33469b97680e8e535fa086, State: Initialized, Role: FOLLOWER
I20260808 02:13:08.763074 24648 ts_tablet_manager.cc:1428] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 02:13:08.763187 24646 consensus_queue.cc:260] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42463 } } peers { permanent_uuid: "ddd5881f97d2462a8f50498e0613c2d6" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 45189 } } peers { permanent_uuid: "969ed5962f33469b97680e8e535fa086" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 43199 } }
I20260808 02:13:08.763548 24626 heartbeater.cc:503] Master 127.18.184.190:36875 was elected leader, sending a full tablet report...
I20260808 02:13:08.763187 24647 consensus_queue.cc:260] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42463 } } peers { permanent_uuid: "ddd5881f97d2462a8f50498e0613c2d6" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 45189 } } peers { permanent_uuid: "969ed5962f33469b97680e8e535fa086" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 43199 } }
I20260808 02:13:08.764209 24348 heartbeater.cc:503] Master 127.18.184.190:36875 was elected leader, sending a full tablet report...
I20260808 02:13:08.764885 24487 heartbeater.cc:503] Master 127.18.184.190:36875 was elected leader, sending a full tablet report...
I20260808 02:13:08.764988 24646 ts_tablet_manager.cc:1428] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 02:13:08.767822 24647 ts_tablet_manager.cc:1428] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
W20260808 02:13:08.821213 24349 tablet.cc:2369] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:08.897758 24654 raft_consensus.cc:482] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:08.898044 24654 raft_consensus.cc:504] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42463 } } peers { permanent_uuid: "ddd5881f97d2462a8f50498e0613c2d6" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 45189 } } peers { permanent_uuid: "969ed5962f33469b97680e8e535fa086" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 43199 } }
I20260808 02:13:08.899199 24654 leader_election.cc:288] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1fca9b7d7994fc0ac07b04f4501ea23 (127.18.184.131:42463), 969ed5962f33469b97680e8e535fa086 (127.18.184.129:43199)
I20260808 02:13:08.901618 24642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.899758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:35497 (local address 127.18.184.131:42463)
0808 02:13:08.899913 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.899920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.899960 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:13:08.900020 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.900027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.900093 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.900241 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.901175 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.901181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.901189 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:08.901201 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.901231 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:08.901272 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.901370 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.901446 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.901488 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:13:08.901618 24490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.899588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:42463 (local address 127.18.184.130:35497)
0808 02:13:08.899731 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:08.899758 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:08.899849 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:08.900255 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:08.900258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:08.900275 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:08.900903 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:08.900907 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:08.900921 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:08.900953 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:08.901362 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:08.901365 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:08.901369 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:08.901445 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:13:08.901485 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 02:13:08.902187 24645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.900118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:41139 (local address 127.18.184.129:43199)
0808 02:13:08.900285 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.900290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.900414 (+ 124us) server_negotiation.cc:407] Connection header received
0808 02:13:08.900518 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.900522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.900563 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.900779 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.901597 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.901600 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.901603 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:08.901614 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.901730 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:08.901750 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.901858 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.902019 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.902066 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:13:08.902199 24655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.899910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:43199 (local address 127.18.184.130:41139)
0808 02:13:08.900310 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:08.900325 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:08.900417 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:08.900809 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:08.900811 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:08.900844 (+ 33us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:08.901406 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:08.901408 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:08.901420 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:08.901456 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:08.901904 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:08.901907 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:08.901910 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:08.901985 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 02:13:08.902018 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":140,"threads_started":1}
I20260808 02:13:08.902370 24579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06301ab250b84b099b3e9a639e9f232e" candidate_uuid: "ddd5881f97d2462a8f50498e0613c2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23" is_pre_election: true
I20260808 02:13:08.902724 24579 raft_consensus.cc:2454] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddd5881f97d2462a8f50498e0613c2d6 in term 0.
I20260808 02:13:08.902866 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06301ab250b84b099b3e9a639e9f232e" candidate_uuid: "ddd5881f97d2462a8f50498e0613c2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "969ed5962f33469b97680e8e535fa086" is_pre_election: true
I20260808 02:13:08.903180 24297 raft_consensus.cc:2454] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddd5881f97d2462a8f50498e0613c2d6 in term 0.
I20260808 02:13:08.903234 24367 leader_election.cc:302] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1fca9b7d7994fc0ac07b04f4501ea23, ddd5881f97d2462a8f50498e0613c2d6; no voters:
I20260808 02:13:08.903491 24654 raft_consensus.cc:2781] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:08.903614 24654 raft_consensus.cc:482] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:08.903683 24654 raft_consensus.cc:3037] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:08.904793 24654 raft_consensus.cc:504] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42463 } } peers { permanent_uuid: "ddd5881f97d2462a8f50498e0613c2d6" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 45189 } } peers { permanent_uuid: "969ed5962f33469b97680e8e535fa086" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 43199 } }
I20260808 02:13:08.905195 24654 leader_election.cc:288] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a1fca9b7d7994fc0ac07b04f4501ea23 (127.18.184.131:42463), 969ed5962f33469b97680e8e535fa086 (127.18.184.129:43199)
I20260808 02:13:08.905556 24579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06301ab250b84b099b3e9a639e9f232e" candidate_uuid: "ddd5881f97d2462a8f50498e0613c2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23"
I20260808 02:13:08.905652 24297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06301ab250b84b099b3e9a639e9f232e" candidate_uuid: "ddd5881f97d2462a8f50498e0613c2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "969ed5962f33469b97680e8e535fa086"
I20260808 02:13:08.905709 24579 raft_consensus.cc:3037] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:08.905813 24297 raft_consensus.cc:3037] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:08.906983 24297 raft_consensus.cc:2454] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddd5881f97d2462a8f50498e0613c2d6 in term 1.
I20260808 02:13:08.907081 24579 raft_consensus.cc:2454] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddd5881f97d2462a8f50498e0613c2d6 in term 1.
I20260808 02:13:08.907334 24367 leader_election.cc:302] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 969ed5962f33469b97680e8e535fa086, ddd5881f97d2462a8f50498e0613c2d6; no voters:
I20260808 02:13:08.907510 24654 raft_consensus.cc:2781] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:08.907816 24654 raft_consensus.cc:686] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 1 LEADER]: Becoming Leader. State: Replica: ddd5881f97d2462a8f50498e0613c2d6, State: Running, Role: LEADER
I20260808 02:13:08.908277 24654 consensus_queue.cc:237] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42463 } } peers { permanent_uuid: "ddd5881f97d2462a8f50498e0613c2d6" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 45189 } } peers { permanent_uuid: "969ed5962f33469b97680e8e535fa086" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 43199 } }
I20260808 02:13:08.912279 24143 catalog_manager.cc:5495] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to ddd5881f97d2462a8f50498e0613c2d6 (127.18.184.130). New cstate: current_term: 1 leader_uuid: "ddd5881f97d2462a8f50498e0613c2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddd5881f97d2462a8f50498e0613c2d6" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 45189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "969ed5962f33469b97680e8e535fa086" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 43199 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:08.937814 24644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.936150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39070 (local address 127.18.184.130:45189)
0808 02:13:08.936301 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.936307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.936338 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:13:08.936488 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.936492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.936564 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.936660 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.937395 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.937399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.937401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:08.937413 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.937434 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:08.937450 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.937548 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.937615 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.937657 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:13:08.943840 24579 raft_consensus.cc:1261] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23 [term 1 FOLLOWER]: Refusing update from remote peer ddd5881f97d2462a8f50498e0613c2d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:08.943832 24297 raft_consensus.cc:1261] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086 [term 1 FOLLOWER]: Refusing update from remote peer ddd5881f97d2462a8f50498e0613c2d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:08.944561 24654 consensus_queue.cc:1035] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1fca9b7d7994fc0ac07b04f4501ea23" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:08.944800 24656 consensus_queue.cc:1035] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "969ed5962f33469b97680e8e535fa086" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 43199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:13:08.950549 24627 tablet.cc:2369] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:08.954902 24662 mvcc.cc:205] Tried to move back new op lower bound from 7316091653906391040 to 7316091653771837440. Current Snapshot: MvccSnapshot[applied={T|T < 7316091653906391040}]
I20260808 02:13:08.963238 24141 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:08.963440 24141 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:08.965935 24141 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fa7e042bf2d4e8f91db6bafda142c7f: Sending DeleteTablet for 3 replicas of tablet 06301ab250b84b099b3e9a639e9f232e
I20260808 02:13:08.966941 24558 tablet_service.cc:1514] Processing DeleteTablet for tablet 06301ab250b84b099b3e9a639e9f232e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:08 UTC) from {username='slave'} at 127.0.0.1:49136
I20260808 02:13:08.967084 24418 tablet_service.cc:1514] Processing DeleteTablet for tablet 06301ab250b84b099b3e9a639e9f232e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:08 UTC) from {username='slave'} at 127.0.0.1:39056
I20260808 02:13:08.967182 24274 tablet_service.cc:1514] Processing DeleteTablet for tablet 06301ab250b84b099b3e9a639e9f232e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:08 UTC) from {username='slave'} at 127.0.0.1:54490
I20260808 02:13:08.967497 24670 tablet_replica.cc:330] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23: stopping tablet replica
I20260808 02:13:08.967738 24672 tablet_replica.cc:330] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086: stopping tablet replica
I20260808 02:13:08.967994 24672 raft_consensus.cc:2229] T 06301ab250b84b099b3e9a639e9f232e P 969ed5962f33469b97680e8e535fa086 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:08.968168 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 24208
I20260808 02:13:08.967911 24670 raft_consensus.cc:2229] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:08.970131 24670 raft_consensus.cc:2258] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:08.971797 24670 ts_tablet_manager.cc:1905] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:08.971933 24671 tablet_replica.cc:330] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6: stopping tablet replica
I20260808 02:13:08.972327 24671 raft_consensus.cc:2229] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:08.972826 24671 raft_consensus.cc:2258] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:08.974431 24671 ts_tablet_manager.cc:1905] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:08.974663 24670 ts_tablet_manager.cc:1918] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:08.974766 24670 log.cc:1196] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/06301ab250b84b099b3e9a639e9f232e
I20260808 02:13:08.975121 24670 ts_tablet_manager.cc:1939] T 06301ab250b84b099b3e9a639e9f232e P a1fca9b7d7994fc0ac07b04f4501ea23: Deleting consensus metadata
I20260808 02:13:08.977200 24671 ts_tablet_manager.cc:1918] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:08.977327 24671 log.cc:1196] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/06301ab250b84b099b3e9a639e9f232e
I20260808 02:13:08.977626 24671 ts_tablet_manager.cc:1939] T 06301ab250b84b099b3e9a639e9f232e P ddd5881f97d2462a8f50498e0613c2d6: Deleting consensus metadata
I20260808 02:13:08.978626 24127 catalog_manager.cc:4841] TS ddd5881f97d2462a8f50498e0613c2d6 (127.18.184.130:45189): tablet 06301ab250b84b099b3e9a639e9f232e (table test-table [id=592afc4b1def4a4fa3544b3e693429c1]) successfully deleted
W20260808 02:13:08.980315 24126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:43199 (error 108)
I20260808 02:13:08.980914 24126 catalog_manager.cc:4841] TS a1fca9b7d7994fc0ac07b04f4501ea23 (127.18.184.131:42463): tablet 06301ab250b84b099b3e9a639e9f232e (table test-table [id=592afc4b1def4a4fa3544b3e693429c1]) successfully deleted
I20260808 02:13:08.981344 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 24351
I20260808 02:13:08.981588 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.981280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:43199 (local address 127.0.0.1:54494)
0808 02:13:08.981437 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:08.981508 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:43199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 02:13:08.983753 24126 catalog_manager.cc:4584] TS 969ed5962f33469b97680e8e535fa086 (127.18.184.129:43199): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 06301ab250b84b099b3e9a639e9f232e: Network error: Client connection negotiation failed: client connection to 127.18.184.129:43199: connect: Connection refused (error 111)
I20260808 02:13:08.988729 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 24491
I20260808 02:13:08.996757 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 24107
2026-08-08T02:13:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1118 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 02:13:09.020617 19170 mini_oidc.cc:152] Starting JWKS server
I20260808 02:13:09.022487 19170 webserver.cc:460] Webserver started at https://127.0.0.1:42399/ using document root <none> and password file <none>
I20260808 02:13:09.022677 19170 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:13:09.022962 19170 webserver.cc:460] Webserver started at http://127.0.0.1:42861/ using document root <none> and password file <none>
2026-08-08T02:13:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:09Z Disabled control of system clock
I20260808 02:13:09.033803 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34595
--webserver_interface=127.18.184.190
--webserver_port=0
--builtin_ntp_servers=127.18.184.148:42963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:34595
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42399/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/testchainca.cert with env {}
W20260808 02:13:09.148393 24679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:09.148716 24679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:09.148805 24679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:09.153657 24679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:09.153739 24679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:09.153760 24679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:09.153784 24679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:09.153829 24679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:13:09.153873 24679 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:13:09.153909 24679 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42399/jwks.json
I20260808 02:13:09.160426 24679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:34595
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42399/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:09.161864 24679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:09.163074 24679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:09.172405 24686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:09.172433 24684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:09.172482 24688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:09.172704 24679 server_base.cc:1034] running on GCE node
I20260808 02:13:09.173368 24679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:09.174492 24679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:09.175681 24679 hybrid_clock.cc:648] HybridClock initialized: now 1786155189175654 us; error 38 us; skew 500 ppm
I20260808 02:13:09.177911 24679 webserver.cc:460] Webserver started at http://127.18.184.190:38867/ using document root <none> and password file <none>
I20260808 02:13:09.178512 24679 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:09.178608 24679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:09.178859 24679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:09.181033 24679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "a0cddc4d1f604961ad9dad9a45f6d24b"
format_stamp: "Formatted at 2026-08-08 02:13:09 on dist-test-slave-9gcw"
server_key: "b087bbcfcf2c1be7ec88b78668471e1d"
server_key_iv: "92f3a60543fe2c6aeee138778b165ab4"
server_key_version: "encryptionkey@0"
I20260808 02:13:09.181702 24679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "a0cddc4d1f604961ad9dad9a45f6d24b"
format_stamp: "Formatted at 2026-08-08 02:13:09 on dist-test-slave-9gcw"
server_key: "b087bbcfcf2c1be7ec88b78668471e1d"
server_key_iv: "92f3a60543fe2c6aeee138778b165ab4"
server_key_version: "encryptionkey@0"
I20260808 02:13:09.185535 24679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:13:09.188031 24695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:09.189502 24679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:09.189656 24679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "a0cddc4d1f604961ad9dad9a45f6d24b"
format_stamp: "Formatted at 2026-08-08 02:13:09 on dist-test-slave-9gcw"
server_key: "b087bbcfcf2c1be7ec88b78668471e1d"
server_key_iv: "92f3a60543fe2c6aeee138778b165ab4"
server_key_version: "encryptionkey@0"
I20260808 02:13:09.189785 24679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:09.212949 24679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:09.213739 24679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:09.213967 24679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:09.221052 24679 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:34595
I20260808 02:13:09.221086 24752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:34595 every 8 connection(s)
I20260808 02:13:09.222103 24679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:09.225353 24753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:09.230754 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 24679
I20260808 02:13:09.230885 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:09.230973 24753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b: Bootstrap starting.
I20260808 02:13:09.231266 19170 external_mini_cluster.cc:1428] Setting key 9aad91e5e50631cdc6a29dac426d3437
I20260808 02:13:09.233752 24753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:09.234679 24753 log.cc:824] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:09.237064 24753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b: No bootstrap required, opened a new log
I20260808 02:13:09.241899 24753 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0cddc4d1f604961ad9dad9a45f6d24b" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34595 } }
I20260808 02:13:09.242127 24756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:09.232632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33944 (local address 127.18.184.190:34595)
0808 02:13:09.233378 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:09.233398 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:09.233441 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:13:09.234356 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:09.234381 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:09.234909 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:09.235268 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:09.235688 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.237111 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.238105 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.238830 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.238998 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:09.239840 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:09.239847 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:09.239867 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:09.240388 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:09.240474 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:09.240500 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:09.240613 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:09.240736 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 02:13:09.241091 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":127,"threads_started":1}
I20260808 02:13:09.242184 24753 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:09.242237 24753 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0cddc4d1f604961ad9dad9a45f6d24b, State: Initialized, Role: FOLLOWER
I20260808 02:13:09.242704 24753 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0cddc4d1f604961ad9dad9a45f6d24b" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34595 } }
I20260808 02:13:09.242863 24753 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:09.242933 24753 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:09.243029 24753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:09.244051 24753 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0cddc4d1f604961ad9dad9a45f6d24b" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34595 } }
I20260808 02:13:09.244410 24753 leader_election.cc:302] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0cddc4d1f604961ad9dad9a45f6d24b; no voters:
I20260808 02:13:09.244764 24753 leader_election.cc:288] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:09.244920 24758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:09.245185 24758 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [term 1 LEADER]: Becoming Leader. State: Replica: a0cddc4d1f604961ad9dad9a45f6d24b, State: Running, Role: LEADER
I20260808 02:13:09.245281 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:13:09.245522 24758 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0cddc4d1f604961ad9dad9a45f6d24b" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34595 } }
I20260808 02:13:09.246107 24753 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:09.247196 24759 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0cddc4d1f604961ad9dad9a45f6d24b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0cddc4d1f604961ad9dad9a45f6d24b" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34595 } } }
I20260808 02:13:09.247329 24759 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:09.247651 24760 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0cddc4d1f604961ad9dad9a45f6d24b. Latest consensus state: current_term: 1 leader_uuid: "a0cddc4d1f604961ad9dad9a45f6d24b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0cddc4d1f604961ad9dad9a45f6d24b" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34595 } } }
I20260808 02:13:09.247745 24760 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:09.247902 24767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:09.251255 24767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:09.257416 24767 catalog_manager.cc:1347] Generated new cluster ID: 241993b99e444563ab0cea3c99e65e6f
I20260808 02:13:09.257504 24767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:09.292145 24767 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:09.293625 24767 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:09.300902 24767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0cddc4d1f604961ad9dad9a45f6d24b: Generated new TSK 0
I20260808 02:13:09.301869 24767 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 02:13:09.340437 24785 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 02:13:09.340628 24785 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 02:13:09.341337 24756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:09.322592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.18.184.190:34595)
0808 02:13:09.322748 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:09.322753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:09.322965 (+ 212us) server_negotiation.cc:407] Connection header received
0808 02:13:09.323170 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:09.323174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:09.323238 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:09.323358 (+ 120us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:13:09.323621 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.324496 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.325487 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.325844 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.325957 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:09.326294 (+ 337us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:13:09.340848 (+ 14554us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42399/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 02:13:09.341184 (+ 336us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33946: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42399/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 02:13:09.341423 24756 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33946: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42399/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 02:13:09.342494 24784 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:09.322445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34595 (local address 127.0.0.1:33946)
0808 02:13:09.322870 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:09.322883 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:09.322995 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:09.323378 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:09.323380 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:09.323387 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:13:09.323501 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:09.323507 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.324637 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.324640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:09.325374 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:09.325384 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.325967 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.325970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:09.326039 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:09.326087 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:13:09.341092 (+ 15005us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42399/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 02:13:09.341357 (+ 265us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.184.190:34595: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42399/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":131,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.184.190:34595: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42399/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 02:13:09.343408 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 24679
2026-08-08T02:13:09Z chronyd exiting
I20260808 02:13:09.383303 19170 test_util.cc:175] -----------------------------------------------
I20260808 02:13:09.383396 19170 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155161841348-19170-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (364 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 02:13:09.384562 19170 mini_oidc.cc:152] Starting JWKS server
I20260808 02:13:09.385602 19170 webserver.cc:460] Webserver started at https://127.0.0.1:39351/ using document root <none> and password file <none>
I20260808 02:13:09.385710 19170 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:13:09.386001 19170 webserver.cc:460] Webserver started at http://127.0.0.1:37759/ using document root <none> and password file <none>
2026-08-08T02:13:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:09Z Disabled control of system clock
I20260808 02:13:09.396479 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:38179
--webserver_interface=127.18.184.190
--webserver_port=0
--builtin_ntp_servers=127.18.184.148:42389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:38179
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39351/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/testchainca.cert with env {}
W20260808 02:13:09.511979 24791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:09.512247 24791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:09.512287 24791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:09.516402 24791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:09.516466 24791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:09.516482 24791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:09.516498 24791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:09.516515 24791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:13:09.516530 24791 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:13:09.516543 24791 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39351/jwks.json
I20260808 02:13:09.522778 24791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:38179
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:38179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39351/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:09.523885 24791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:09.524909 24791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:09.534061 24797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:09.534070 24800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:09.534114 24791 server_base.cc:1034] running on GCE node
W20260808 02:13:09.534082 24798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:09.535187 24791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:09.536463 24791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:09.537634 24791 hybrid_clock.cc:648] HybridClock initialized: now 1786155189537610 us; error 40 us; skew 500 ppm
I20260808 02:13:09.539635 24791 webserver.cc:460] Webserver started at http://127.18.184.190:45669/ using document root <none> and password file <none>
I20260808 02:13:09.540237 24791 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:09.540303 24791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:09.540525 24791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:09.542863 24791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "b0d74e4d234648c682fed418c0923eab"
format_stamp: "Formatted at 2026-08-08 02:13:09 on dist-test-slave-9gcw"
server_key: "696d9f24425f6e5d41a7b680ec70c0d3"
server_key_iv: "54a5d1a5f32812ccd8b2af6ea797f18f"
server_key_version: "encryptionkey@0"
I20260808 02:13:09.543483 24791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "b0d74e4d234648c682fed418c0923eab"
format_stamp: "Formatted at 2026-08-08 02:13:09 on dist-test-slave-9gcw"
server_key: "696d9f24425f6e5d41a7b680ec70c0d3"
server_key_iv: "54a5d1a5f32812ccd8b2af6ea797f18f"
server_key_version: "encryptionkey@0"
I20260808 02:13:09.547453 24791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 02:13:09.549965 24807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:09.551224 24791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:09.551378 24791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "b0d74e4d234648c682fed418c0923eab"
format_stamp: "Formatted at 2026-08-08 02:13:09 on dist-test-slave-9gcw"
server_key: "696d9f24425f6e5d41a7b680ec70c0d3"
server_key_iv: "54a5d1a5f32812ccd8b2af6ea797f18f"
server_key_version: "encryptionkey@0"
I20260808 02:13:09.551496 24791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:09.567003 24791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:09.567768 24791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:09.567992 24791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:09.575722 24791 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:38179
I20260808 02:13:09.575729 24860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:38179 every 8 connection(s)
I20260808 02:13:09.576707 24791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:09.580031 24862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:09.582921 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 24791
I20260808 02:13:09.583062 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:09.583345 19170 external_mini_cluster.cc:1428] Setting key 4347b50e687544776b8d9caac65aeaf9
I20260808 02:13:09.586426 24862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab: Bootstrap starting.
I20260808 02:13:09.588788 24862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:09.589618 24862 log.cc:824] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:09.591601 24862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab: No bootstrap required, opened a new log
I20260808 02:13:09.592780 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:09.584773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.18.184.190:38179)
0808 02:13:09.585514 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:09.585544 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:09.585580 (+ 36us) server_negotiation.cc:407] Connection header received
0808 02:13:09.586245 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:09.586273 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:09.586730 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:09.587068 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:09.587510 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.588496 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.589520 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.590056 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.590229 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:09.591106 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:09.591112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:09.591124 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:09.591513 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:09.591570 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:09.591590 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:09.591698 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:09.591826 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 02:13:09.592082 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":133,"threads_started":1}
I20260808 02:13:09.595394 24862 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d74e4d234648c682fed418c0923eab" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38179 } }
I20260808 02:13:09.595726 24862 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:09.595814 24862 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0d74e4d234648c682fed418c0923eab, State: Initialized, Role: FOLLOWER
I20260808 02:13:09.596264 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:13:09.596304 24862 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d74e4d234648c682fed418c0923eab" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38179 } }
I20260808 02:13:09.596496 24862 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:09.596560 24862 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:09.596681 24862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:09.597831 24862 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d74e4d234648c682fed418c0923eab" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38179 } }
I20260808 02:13:09.598174 24862 leader_election.cc:302] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0d74e4d234648c682fed418c0923eab; no voters:
I20260808 02:13:09.598495 24862 leader_election.cc:288] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:09.598594 24868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:09.598788 24868 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [term 1 LEADER]: Becoming Leader. State: Replica: b0d74e4d234648c682fed418c0923eab, State: Running, Role: LEADER
I20260808 02:13:09.599116 24868 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d74e4d234648c682fed418c0923eab" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38179 } }
I20260808 02:13:09.599834 24862 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:09.600772 24869 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0d74e4d234648c682fed418c0923eab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d74e4d234648c682fed418c0923eab" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38179 } } }
I20260808 02:13:09.600926 24869 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:09.601208 24870 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0d74e4d234648c682fed418c0923eab. Latest consensus state: current_term: 1 leader_uuid: "b0d74e4d234648c682fed418c0923eab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d74e4d234648c682fed418c0923eab" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38179 } } }
I20260808 02:13:09.601312 24870 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab [sys.catalog]: This master's current role is: LEADER
W20260808 02:13:09.604766 24885 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:13:09.604851 24885 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:13:09.604933 24878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:09.605711 24878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:09.611105 24878 catalog_manager.cc:1347] Generated new cluster ID: af56c6b265d0476c820f71a206f27dcd
I20260808 02:13:09.611192 24878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:09.639528 24878 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:09.640983 24878 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:09.660432 24878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0d74e4d234648c682fed418c0923eab: Generated new TSK 0
I20260808 02:13:09.661422 24878 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 02:13:09.690620 24895 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 02:13:09.690726 24895 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 02:13:09.691480 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:09.671237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59780 (local address 127.18.184.190:38179)
0808 02:13:09.671397 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:09.671403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:09.671660 (+ 257us) server_negotiation.cc:407] Connection header received
0808 02:13:09.671817 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:09.671819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:09.671872 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:09.671969 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:13:09.672274 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.672833 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.674156 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.674400 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.674483 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:09.674834 (+ 351us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:13:09.691082 (+ 16248us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 02:13:09.691358 (+ 276us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59780: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 02:13:09.691571 24866 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59780: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 02:13:09.691768 24894 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:09.671184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:38179 (local address 127.0.0.1:59780)
0808 02:13:09.671531 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:09.671546 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:09.671672 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:09.671990 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:09.671993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:09.672001 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:13:09.672119 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:09.672126 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.672966 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.672970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:09.674010 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:09.674022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.674520 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.674524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:09.674612 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:09.674641 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:13:09.691336 (+ 16695us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 02:13:09.691551 (+ 215us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.184.190:38179: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":132,"threads_started":1}
I20260808 02:13:09.692435 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 24791
2026-08-08T02:13:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (351 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 02:13:09.735625 19170 mini_oidc.cc:152] Starting JWKS server
I20260808 02:13:09.736487 19170 webserver.cc:460] Webserver started at https://127.0.0.1:45933/ using document root <none> and password file <none>
I20260808 02:13:09.736570 19170 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:13:09.736829 19170 webserver.cc:460] Webserver started at http://127.0.0.1:35523/ using document root <none> and password file <none>
2026-08-08T02:13:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:09Z Disabled control of system clock
I20260808 02:13:09.747401 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:36793
--webserver_interface=127.18.184.190
--webserver_port=0
--builtin_ntp_servers=127.18.184.148:40291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:36793
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45933/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 02:13:09.865198 24902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:09.865518 24902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:09.865559 24902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:09.869728 24902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:09.869812 24902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:09.869828 24902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:09.869846 24902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:09.869863 24902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:13:09.869879 24902 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:13:09.869894 24902 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45933/jwks.json
I20260808 02:13:09.874888 24902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:40291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:36793
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:36793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45933/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:09.876065 24902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:09.877573 24902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:09.886574 24909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:09.886606 24912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:09.886574 24910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:09.886709 24902 server_base.cc:1034] running on GCE node
I20260808 02:13:09.887504 24902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:09.888675 24902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:09.889882 24902 hybrid_clock.cc:648] HybridClock initialized: now 1786155189889871 us; error 30 us; skew 500 ppm
I20260808 02:13:09.891896 24902 webserver.cc:460] Webserver started at http://127.18.184.190:35503/ using document root <none> and password file <none>
I20260808 02:13:09.892516 24902 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:09.892617 24902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:09.892875 24902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:09.895205 24902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "e1f24d0ce9a546baa93276a0c4ae5393"
format_stamp: "Formatted at 2026-08-08 02:13:09 on dist-test-slave-9gcw"
server_key: "d7ef3f0ae7b0e6ffe66817ab462e5663"
server_key_iv: "36edc439f34a475eef56255aeeedae63"
server_key_version: "encryptionkey@0"
I20260808 02:13:09.895887 24902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "e1f24d0ce9a546baa93276a0c4ae5393"
format_stamp: "Formatted at 2026-08-08 02:13:09 on dist-test-slave-9gcw"
server_key: "d7ef3f0ae7b0e6ffe66817ab462e5663"
server_key_iv: "36edc439f34a475eef56255aeeedae63"
server_key_version: "encryptionkey@0"
I20260808 02:13:09.899906 24902 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 02:13:09.902489 24918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:09.903754 24902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:09.903905 24902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "e1f24d0ce9a546baa93276a0c4ae5393"
format_stamp: "Formatted at 2026-08-08 02:13:09 on dist-test-slave-9gcw"
server_key: "d7ef3f0ae7b0e6ffe66817ab462e5663"
server_key_iv: "36edc439f34a475eef56255aeeedae63"
server_key_version: "encryptionkey@0"
I20260808 02:13:09.904022 24902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:09.928675 24902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:09.929518 24902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:09.929733 24902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:09.937716 24902 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:36793
I20260808 02:13:09.937765 24971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:36793 every 8 connection(s)
I20260808 02:13:09.938711 24902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:09.942109 24972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:09.944463 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 24902
I20260808 02:13:09.944607 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:09.944895 19170 external_mini_cluster.cc:1428] Setting key fdc51520cd9accd5cc423d816c047c49
I20260808 02:13:09.948227 24972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393: Bootstrap starting.
I20260808 02:13:09.950639 24972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:09.951642 24972 log.cc:824] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:09.954118 24972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393: No bootstrap required, opened a new log
I20260808 02:13:09.954880 24975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:09.946364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47402 (local address 127.18.184.190:36793)
0808 02:13:09.947030 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:09.947060 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:09.947095 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:13:09.947692 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:09.947707 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:09.948105 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:09.948440 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:09.948891 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.949935 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.950877 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.951605 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.951814 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:09.952763 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:09.952770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:09.952786 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:09.953305 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:09.953389 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:09.953417 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:09.953514 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:09.953630 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 02:13:09.953909 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":167,"threads_started":1}
I20260808 02:13:09.957831 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:13:09.958485 24972 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f24d0ce9a546baa93276a0c4ae5393" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36793 } }
I20260808 02:13:09.958736 24972 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:09.958796 24972 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f24d0ce9a546baa93276a0c4ae5393, State: Initialized, Role: FOLLOWER
I20260808 02:13:09.959265 24972 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f24d0ce9a546baa93276a0c4ae5393" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36793 } }
I20260808 02:13:09.959422 24972 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:09.959517 24972 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:09.959641 24972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:09.960630 24972 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f24d0ce9a546baa93276a0c4ae5393" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36793 } }
I20260808 02:13:09.960985 24972 leader_election.cc:302] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1f24d0ce9a546baa93276a0c4ae5393; no voters:
I20260808 02:13:09.961352 24972 leader_election.cc:288] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:09.961476 24977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:09.961735 24977 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [term 1 LEADER]: Becoming Leader. State: Replica: e1f24d0ce9a546baa93276a0c4ae5393, State: Running, Role: LEADER
I20260808 02:13:09.962071 24977 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f24d0ce9a546baa93276a0c4ae5393" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36793 } }
I20260808 02:13:09.962739 24972 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:09.963856 24980 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1f24d0ce9a546baa93276a0c4ae5393. Latest consensus state: current_term: 1 leader_uuid: "e1f24d0ce9a546baa93276a0c4ae5393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f24d0ce9a546baa93276a0c4ae5393" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36793 } } }
I20260808 02:13:09.963999 24980 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:09.964422 24987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:09.964895 24979 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1f24d0ce9a546baa93276a0c4ae5393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f24d0ce9a546baa93276a0c4ae5393" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36793 } } }
I20260808 02:13:09.965018 24979 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:09.967741 24987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:09.973577 24987 catalog_manager.cc:1347] Generated new cluster ID: a635d5302806490ab77c65c78ced7149
I20260808 02:13:09.973654 24987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:09.984912 24987 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:09.985920 24987 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:09.992654 24987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1f24d0ce9a546baa93276a0c4ae5393: Generated new TSK 0
I20260808 02:13:09.993495 24987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:10.005993 24975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:10.000647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47408 (local address 127.18.184.190:36793)
0808 02:13:10.000793 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:10.000798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:10.000945 (+ 147us) server_negotiation.cc:407] Connection header received
0808 02:13:10.001108 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:10.001110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:10.001213 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:10.001306 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:13:10.001623 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.002277 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.003574 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.003804 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.003897 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.004260 (+ 363us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:13:10.005583 (+ 1323us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 02:13:10.005861 (+ 278us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47408: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 02:13:10.006084 24975 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47408: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 02:13:10.008868 25002 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:10.000508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:36793 (local address 127.0.0.1:47408)
0808 02:13:10.000848 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:10.000863 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:10.000984 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:10.001336 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:10.001339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:10.001348 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:13:10.001475 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:10.001482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.002405 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.002409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:10.003432 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:10.003442 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.003946 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.003950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:10.004042 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.004077 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:13:10.005749 (+ 1672us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 02:13:10.008677 (+ 2928us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.184.190:36793: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45933/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":2732,"thread_start_us":99,"threads_started":1}
I20260808 02:13:10.009583 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 24902
E20260808 02:13:10.019860 25003 webserver.cc:533] Webserver: sslize failed:
E20260808 02:13:10.019943 25003 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T02:13:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (316 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T02:13:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:10Z Disabled control of system clock
I20260808 02:13:10.063654 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:44737
--webserver_interface=127.18.184.190
--webserver_port=0
--builtin_ntp_servers=127.18.184.148:37359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:44737
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/testchainca.cert with env {}
W20260808 02:13:10.181370 25007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:10.181699 25007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:10.181773 25007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:10.185822 25007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:10.185925 25007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:10.185967 25007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:10.186003 25007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:10.186036 25007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:13:10.186085 25007 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:13:10.186118 25007 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 02:13:10.191923 25007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:37359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:44737
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:44737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:10.193229 25007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:10.194329 25007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:10.203303 25013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:10.203310 25015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:10.203303 25012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:10.203663 25007 server_base.cc:1034] running on GCE node
I20260808 02:13:10.204315 25007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:10.205569 25007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:10.206753 25007 hybrid_clock.cc:648] HybridClock initialized: now 1786155190206742 us; error 29 us; skew 500 ppm
I20260808 02:13:10.208757 25007 webserver.cc:460] Webserver started at http://127.18.184.190:34661/ using document root <none> and password file <none>
I20260808 02:13:10.209393 25007 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:10.209461 25007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:10.209630 25007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:10.211771 25007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "201c87cfe40e4e6db83d9d0f801864ce"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-9gcw"
server_key: "687165c5216828aefc6052ecbbf395fd"
server_key_iv: "57e5aa57141759ccbae599cc3354ead7"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.212411 25007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "201c87cfe40e4e6db83d9d0f801864ce"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-9gcw"
server_key: "687165c5216828aefc6052ecbbf395fd"
server_key_iv: "57e5aa57141759ccbae599cc3354ead7"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.216454 25007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:13:10.219060 25021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:10.220521 25007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:13:10.220664 25007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "201c87cfe40e4e6db83d9d0f801864ce"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-9gcw"
server_key: "687165c5216828aefc6052ecbbf395fd"
server_key_iv: "57e5aa57141759ccbae599cc3354ead7"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.220794 25007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:10.239648 25007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:10.240485 25007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:10.240698 25007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:10.247996 25007 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:44737
I20260808 02:13:10.248004 25077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:44737 every 8 connection(s)
I20260808 02:13:10.249107 25007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:10.250411 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 25007
I20260808 02:13:10.250554 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:10.250877 19170 external_mini_cluster.cc:1428] Setting key 425b4fef0b420284d64a78c691d9bfd7
I20260808 02:13:10.252436 25078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:10.259734 25078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce: Bootstrap starting.
I20260808 02:13:10.262094 25081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:10.252448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.18.184.190:44737)
0808 02:13:10.253248 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:10.253269 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:10.253323 (+ 54us) server_negotiation.cc:407] Connection header received
0808 02:13:10.254208 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:10.254232 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:10.254752 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:10.255139 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:10.255622 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.257080 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.258053 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.258784 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.258950 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.259858 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:10.259865 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:10.259881 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:10.260397 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:10.260472 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:10.260500 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:10.260599 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:10.260757 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 02:13:10.261047 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":160,"threads_started":1}
I20260808 02:13:10.262833 25078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:10.263669 25078 log.cc:824] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:10.265393 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:13:10.266160 25078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce: No bootstrap required, opened a new log
I20260808 02:13:10.270090 25078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "201c87cfe40e4e6db83d9d0f801864ce" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44737 } }
I20260808 02:13:10.270386 25078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:10.270473 25078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 201c87cfe40e4e6db83d9d0f801864ce, State: Initialized, Role: FOLLOWER
I20260808 02:13:10.270944 25078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "201c87cfe40e4e6db83d9d0f801864ce" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44737 } }
I20260808 02:13:10.271098 25078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:10.271189 25078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:10.271314 25078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:10.272361 25078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "201c87cfe40e4e6db83d9d0f801864ce" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44737 } }
I20260808 02:13:10.272770 25078 leader_election.cc:302] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 201c87cfe40e4e6db83d9d0f801864ce; no voters:
I20260808 02:13:10.273197 25078 leader_election.cc:288] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:10.273383 25083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:10.273672 25083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [term 1 LEADER]: Becoming Leader. State: Replica: 201c87cfe40e4e6db83d9d0f801864ce, State: Running, Role: LEADER
I20260808 02:13:10.274008 25083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "201c87cfe40e4e6db83d9d0f801864ce" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44737 } }
I20260808 02:13:10.274555 25078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:10.275882 25084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "201c87cfe40e4e6db83d9d0f801864ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "201c87cfe40e4e6db83d9d0f801864ce" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44737 } } }
I20260808 02:13:10.276033 25084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:10.275900 25086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 201c87cfe40e4e6db83d9d0f801864ce. Latest consensus state: current_term: 1 leader_uuid: "201c87cfe40e4e6db83d9d0f801864ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "201c87cfe40e4e6db83d9d0f801864ce" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44737 } } }
I20260808 02:13:10.276125 25086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce [sys.catalog]: This master's current role is: LEADER
W20260808 02:13:10.279366 25099 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:13:10.279459 25099 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:13:10.279560 25094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:10.280244 25094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:10.286062 25094 catalog_manager.cc:1347] Generated new cluster ID: b4b9a9e5949c430eb8b999c7249fcc48
I20260808 02:13:10.286136 25094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:10.303750 25094 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:10.305176 25094 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:10.313387 25094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 201c87cfe40e4e6db83d9d0f801864ce: Generated new TSK 0
I20260808 02:13:10.314348 25094 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:11.207074 25081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:10.330678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.18.184.190:44737)
0808 02:13:10.330843 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:10.330848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:10.331094 (+ 246us) server_negotiation.cc:407] Connection header received
0808 02:13:10.331335 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:10.331338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:10.331391 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:10.331517 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:13:10.331793 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.332468 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.333912 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.334142 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.334269 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.334707 (+ 438us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:13:11.206582 (+871875us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 02:13:11.206945 (+ 363us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33626: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 02:13:11.207196 25081 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33626: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 02:13:11.207247 25108 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:10.330579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:44737 (local address 127.0.0.1:33626)
0808 02:13:10.330988 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:10.331008 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:10.331168 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:10.331517 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:10.331521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:10.331530 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:13:10.331653 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:10.331660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.332662 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.332668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:10.333767 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:10.333779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.334297 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.334301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:10.334407 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.334440 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:13:11.206891 (+872451us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 02:13:11.207090 (+ 199us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.184.190:44737: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":130,"threads_started":1}
I20260808 02:13:11.207947 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 25007
2026-08-08T02:13:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1177 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (131 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (131 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:11 dist-test-slave-9gcw krb5kdc[25120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:11 dist-test-slave-9gcw krb5kdc[25120](info): set up 2 sockets
Aug 08 02:13:11 dist-test-slave-9gcw krb5kdc[25120](info): commencing operation
krb5kdc: starting...
W20260808 02:13:12.613652 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:13:12 dist-test-slave-9gcw krb5kdc[25120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:13:12.746948 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:40527
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:40527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:12.875119 25134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:12.875435 25134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:12.875483 25134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:12.879580 25134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:12.879662 25134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:12.879686 25134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:12.879706 25134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:12.879724 25134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:12.885278 25134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:40527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:40527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:12.886410 25134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:12.887480 25134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:12.896888 25143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:12.896905 25139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:12.896957 25134 server_base.cc:1034] running on GCE node
W20260808 02:13:12.897046 25141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:12.897743 25134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:12.899008 25134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:12.900194 25134 hybrid_clock.cc:648] HybridClock initialized: now 1786155192900170 us; error 49 us; skew 500 ppm
Aug 08 02:13:12 dist-test-slave-9gcw krb5kdc[25120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155192, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:12.907294 25134 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:13:12.908422 25134 webserver.cc:460] Webserver started at http://127.18.184.190:46111/ using document root <none> and password file <none>
I20260808 02:13:12.909055 25134 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:12.909178 25134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:12.909428 25134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:12.911748 25134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "f8e8e57280d24941b64ee213064e8e97"
format_stamp: "Formatted at 2026-08-08 02:13:12 on dist-test-slave-9gcw"
server_key: "e4a26ce13aa19da1ac0f49ebaa997c71"
server_key_iv: "50538e0c3445eba491d822b3e199e790"
server_key_version: "encryptionkey@0"
I20260808 02:13:12.912390 25134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "f8e8e57280d24941b64ee213064e8e97"
format_stamp: "Formatted at 2026-08-08 02:13:12 on dist-test-slave-9gcw"
server_key: "e4a26ce13aa19da1ac0f49ebaa997c71"
server_key_iv: "50538e0c3445eba491d822b3e199e790"
server_key_version: "encryptionkey@0"
I20260808 02:13:12.916491 25134 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:13:12.919148 25150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:12.920521 25134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:12.920753 25134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "f8e8e57280d24941b64ee213064e8e97"
format_stamp: "Formatted at 2026-08-08 02:13:12 on dist-test-slave-9gcw"
server_key: "e4a26ce13aa19da1ac0f49ebaa997c71"
server_key_iv: "50538e0c3445eba491d822b3e199e790"
server_key_version: "encryptionkey@0"
I20260808 02:13:12.920894 25134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:12.941793 25134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:12.944805 25134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:12.944964 25134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:12.952378 25134 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:40527
I20260808 02:13:12.952412 25207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:40527 every 8 connection(s)
I20260808 02:13:12.953451 25134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:12.955843 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 25134
I20260808 02:13:12.955951 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:12.956292 19170 external_mini_cluster.cc:1428] Setting key ce8846cb108bb78b862563c180b3565b
I20260808 02:13:12.956521 25208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:12.964214 25208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97: Bootstrap starting.
Aug 08 02:13:12 dist-test-slave-9gcw krb5kdc[25120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:12.966955 25208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:12.967869 25208 log.cc:824] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:12.970533 25208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97: No bootstrap required, opened a new log
I20260808 02:13:12.973304 25211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:12.958049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.18.184.190:40527)
0808 02:13:12.958496 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:12.958517 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:12.958561 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:13:12.959493 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:12.959518 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:12.959957 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:12.960342 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:12.961073 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:12.962605 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:12.963560 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:12.964250 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:12.964425 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:12.967662 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:12.967668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:12.967685 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:12.967736 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:12.969880 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:12.970396 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:12.970402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:12.970407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:12.970547 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:12.970870 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:12.970873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:12.970875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:12.971389 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:12.971585 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:12.971883 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 02:13:12.972187 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":126,"threads_started":1}
I20260808 02:13:12.975199 25208 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e57280d24941b64ee213064e8e97" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40527 } }
I20260808 02:13:12.975494 25208 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:12.975622 25208 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e8e57280d24941b64ee213064e8e97, State: Initialized, Role: FOLLOWER
I20260808 02:13:12.976092 25208 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e57280d24941b64ee213064e8e97" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40527 } }
I20260808 02:13:12.976228 25208 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:12.976280 25208 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:12.976352 25208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:12.977339 25208 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e57280d24941b64ee213064e8e97" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40527 } }
I20260808 02:13:12.977635 25208 leader_election.cc:302] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8e8e57280d24941b64ee213064e8e97; no voters:
I20260808 02:13:12.977892 25208 leader_election.cc:288] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:12.978120 25213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:12.978389 25213 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [term 1 LEADER]: Becoming Leader. State: Replica: f8e8e57280d24941b64ee213064e8e97, State: Running, Role: LEADER
I20260808 02:13:12.978760 25213 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e57280d24941b64ee213064e8e97" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40527 } }
I20260808 02:13:12.979192 25208 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:12.980399 25215 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8e8e57280d24941b64ee213064e8e97. Latest consensus state: current_term: 1 leader_uuid: "f8e8e57280d24941b64ee213064e8e97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e57280d24941b64ee213064e8e97" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40527 } } }
I20260808 02:13:12.980526 25215 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:12.980710 25214 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8e8e57280d24941b64ee213064e8e97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e57280d24941b64ee213064e8e97" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40527 } } }
I20260808 02:13:12.980876 25214 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:12.981321 25225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:12.984095 25225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:12.989610 25225 catalog_manager.cc:1347] Generated new cluster ID: 026106eccca64861ac002c89e5076a71
I20260808 02:13:12.989708 25225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:12.999095 25225 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:13.000362 25225 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:13.011186 25225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8e8e57280d24941b64ee213064e8e97: Generated new TSK 0
I20260808 02:13:13.012089 25225 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:13:13.048702 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40527
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:13.167609 25236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:13.167901 25236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:13.167948 25236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:13.172083 25236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:13.172154 25236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:13.172226 25236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:13.177649 25236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:13.178745 25236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:13.179854 25236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:13.190018 25245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:13.190032 25242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:13.190246 25236 server_base.cc:1034] running on GCE node
W20260808 02:13:13.190390 25241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:13.190841 25236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:13.191433 25236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:13.192708 25236 hybrid_clock.cc:648] HybridClock initialized: now 1786155193192691 us; error 30 us; skew 500 ppm
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.196036 25236 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:13:13.197117 25236 webserver.cc:460] Webserver started at http://127.18.184.129:37975/ using document root <none> and password file <none>
I20260808 02:13:13.197748 25236 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:13.197810 25236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:13.198058 25236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:13.200176 25236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "65ccf42cd57d42f789f5b79c34ceb6d5"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-9gcw"
server_key: "7fb3b49e4191e51041a5bef7817a88ab"
server_key_iv: "bdd95d2d0a1a19d900701a9a04773406"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.200798 25236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "65ccf42cd57d42f789f5b79c34ceb6d5"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-9gcw"
server_key: "7fb3b49e4191e51041a5bef7817a88ab"
server_key_iv: "bdd95d2d0a1a19d900701a9a04773406"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.204826 25236 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 02:13:13.207484 25253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.208853 25236 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:13.208959 25236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "65ccf42cd57d42f789f5b79c34ceb6d5"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-9gcw"
server_key: "7fb3b49e4191e51041a5bef7817a88ab"
server_key_iv: "bdd95d2d0a1a19d900701a9a04773406"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.209096 25236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:13.221069 25236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:13.224193 25236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:13.224402 25236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:13.225011 25236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:13.225957 25236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:13.226006 25236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.226073 25236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:13.226104 25236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.235648 25236 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:36399
I20260808 02:13:13.235669 25377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:36399 every 8 connection(s)
I20260808 02:13:13.236572 25236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:13.246840 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 25236
I20260808 02:13:13.247001 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:13.247288 19170 external_mini_cluster.cc:1428] Setting key 55999eb46bbbcf3a6b8f94ddab50a281
I20260808 02:13:13.251621 25211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.238473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:43687 (local address 127.18.184.190:40527)
0808 02:13:13.238655 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.238659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.239519 (+ 860us) server_negotiation.cc:407] Connection header received
0808 02:13:13.240891 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.240894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.240947 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.241037 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.242324 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.242942 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.244088 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.244321 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.244402 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.246948 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:13.246957 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:13.246959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:13.247011 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:13.248863 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.249524 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.249527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.249528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.249616 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.250198 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.250201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.250202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.250397 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:13.250476 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.251217 (+ 741us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.251389 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 02:13:13.252559 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.238851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40527 (local address 127.18.184.129:43687)
0808 02:13:13.239341 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:13.239391 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:13.240665 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:13.241296 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:13.241303 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:13.241673 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:13.242155 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.242169 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.243063 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.243065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.243965 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.243975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.244461 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.244464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.244606 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.245238 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:13.245254 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:13.246764 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:13.248970 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:13.248976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:13.248992 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:13.249412 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:13.249766 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:13.249772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:13.249777 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:13.249998 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:13.250519 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:13.250529 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:13.250907 (+ 378us) client_negotiation.cc:769] Sending connection context
0808 02:13:13.251143 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 02:13:13.251559 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":112,"threads_started":1}
I20260808 02:13:13.253935 25378 heartbeater.cc:347] Connected to a master server at 127.18.184.190:40527
I20260808 02:13:13.254192 25378 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:13.254839 25378 heartbeater.cc:511] Master 127.18.184.190:40527 requested a full tablet report, sending...
I20260808 02:13:13.256450 25168 ts_manager.cc:194] Registered new tserver with Master: 65ccf42cd57d42f789f5b79c34ceb6d5 (127.18.184.129:36399)
I20260808 02:13:13.257751 25168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:43687
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:13:13.281764 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40527
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:13.398630 25383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:13.398932 25383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:13.399009 25383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:13.403098 25383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:13.403208 25383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:13.403311 25383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:13.408854 25383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:13.410029 25383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:13.411124 25383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:13.421434 25391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:13.421451 25388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:13.421469 25389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:13.421617 25383 server_base.cc:1034] running on GCE node
I20260808 02:13:13.422104 25383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:13.422766 25383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:13.423931 25383 hybrid_clock.cc:648] HybridClock initialized: now 1786155193423902 us; error 38 us; skew 500 ppm
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.427014 25383 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:13:13.427917 25383 webserver.cc:460] Webserver started at http://127.18.184.130:43531/ using document root <none> and password file <none>
I20260808 02:13:13.428484 25383 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:13.428550 25383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:13.428771 25383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:13.431077 25383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "193cb73bb9b1464a8e22644e1dd05dbc"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-9gcw"
server_key: "8b89d664eeb8a20d74ba11803ed9e920"
server_key_iv: "33c5bff9267f635ae1392fdc1a3e47de"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.431643 25383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "193cb73bb9b1464a8e22644e1dd05dbc"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-9gcw"
server_key: "8b89d664eeb8a20d74ba11803ed9e920"
server_key_iv: "33c5bff9267f635ae1392fdc1a3e47de"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.435576 25383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260808 02:13:13.438163 25399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.439641 25383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:13.439786 25383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "193cb73bb9b1464a8e22644e1dd05dbc"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-9gcw"
server_key: "8b89d664eeb8a20d74ba11803ed9e920"
server_key_iv: "33c5bff9267f635ae1392fdc1a3e47de"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.439904 25383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:13.462606 25383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:13.466248 25383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:13.466475 25383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:13.467154 25383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:13.468066 25383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:13.468137 25383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.468212 25383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:13.468254 25383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.478029 25383 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:33045
I20260808 02:13:13.478055 25521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:33045 every 8 connection(s)
I20260808 02:13:13.479028 25383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:13.479813 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 25383
I20260808 02:13:13.479926 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:13.480264 19170 external_mini_cluster.cc:1428] Setting key a1a3fc4ec49288275e903baa14f3c30a
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
I20260808 02:13:13.496973 25211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.481249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:46121 (local address 127.18.184.190:40527)
0808 02:13:13.481399 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.481404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.482495 (+ 1091us) server_negotiation.cc:407] Connection header received
0808 02:13:13.484803 (+ 2308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.484808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.484876 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.484979 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.486458 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.487389 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.488803 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.489279 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.489414 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.492451 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:13.492454 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:13.492456 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:13.492482 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:13.494078 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.494920 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.494923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.494925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.495049 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.495493 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.495497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.495499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.495803 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:13.495926 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.496489 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.496686 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:13:13.497822 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.481712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40527 (local address 127.18.184.130:46121)
0808 02:13:13.482306 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:13.482375 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:13.484468 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:13.485202 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:13.485211 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:13.485735 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:13.486286 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.486301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.487550 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.487554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.488579 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.488589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.489449 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.489452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.489701 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.490384 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:13.490401 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:13.492264 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:13.494218 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:13.494223 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:13.494236 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:13.494627 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:13.495177 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:13.495181 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:13.495183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:13.495392 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:13.495935 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:13.495940 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:13.496212 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 02:13:13.496402 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 02:13:13.496702 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":125,"threads_started":1}
I20260808 02:13:13.499418 25522 heartbeater.cc:347] Connected to a master server at 127.18.184.190:40527
I20260808 02:13:13.499691 25522 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:13.500238 25522 heartbeater.cc:511] Master 127.18.184.190:40527 requested a full tablet report, sending...
I20260808 02:13:13.501437 25168 ts_manager.cc:194] Registered new tserver with Master: 193cb73bb9b1464a8e22644e1dd05dbc (127.18.184.130:33045)
I20260808 02:13:13.502089 25168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:46121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:13:13.516222 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40527
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:13.635461 25529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:13.635762 25529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:13.635810 25529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:13.639993 25529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:13.640079 25529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:13.640141 25529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:13.646253 25529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:13.647408 25529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:13.648456 25529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:13.658756 25535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:13.658941 25529 server_base.cc:1034] running on GCE node
W20260808 02:13:13.658726 25538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:13.658795 25536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:13.659596 25529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:13.660424 25529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:13.661636 25529 hybrid_clock.cc:648] HybridClock initialized: now 1786155193661605 us; error 44 us; skew 500 ppm
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.665055 25529 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:13:13.666049 25529 webserver.cc:460] Webserver started at http://127.18.184.131:38985/ using document root <none> and password file <none>
I20260808 02:13:13.666656 25529 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:13.666749 25529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:13.666986 25529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:13.669072 25529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "f38f6ab4b3be43f997237f5174640a32"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-9gcw"
server_key: "051a3dcb7c3369ac03839ddae0ae69ba"
server_key_iv: "1174133f60f27ee8f1dda936341168df"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.669703 25529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "f38f6ab4b3be43f997237f5174640a32"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-9gcw"
server_key: "051a3dcb7c3369ac03839ddae0ae69ba"
server_key_iv: "1174133f60f27ee8f1dda936341168df"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.673717 25529 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:13:13.676311 25545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.677793 25529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:13.677927 25529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "f38f6ab4b3be43f997237f5174640a32"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-9gcw"
server_key: "051a3dcb7c3369ac03839ddae0ae69ba"
server_key_iv: "1174133f60f27ee8f1dda936341168df"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.678040 25529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:13.693240 25529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:13.696623 25529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:13.696849 25529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:13.697604 25529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:13.698483 25529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:13.698531 25529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.698571 25529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:13.698587 25529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.708035 25529 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:36305
I20260808 02:13:13.708089 25668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:36305 every 8 connection(s)
I20260808 02:13:13.709046 25529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:13.714489 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 25529
I20260808 02:13:13.714630 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:13.714956 19170 external_mini_cluster.cc:1428] Setting key 2f3017e15619438629a9b7f0ca844390
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:13.724290 25211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.711120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:42435 (local address 127.18.184.190:40527)
0808 02:13:13.711319 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.711325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.712276 (+ 951us) server_negotiation.cc:407] Connection header received
0808 02:13:13.713925 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.713927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.713977 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.714050 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.715412 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.716109 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.717435 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.717711 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.717838 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.720565 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:13.720569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:13.720573 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:13.720629 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:13.722173 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.722712 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.722716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.722718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.722805 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.723151 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.723157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.723162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.723362 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:13.723477 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.723982 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.724117 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 02:13:13.725400 25672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.711533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40527 (local address 127.18.184.131:42435)
0808 02:13:13.712105 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:13.712159 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:13.713700 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:13.714193 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:13.714199 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:13.714588 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:13.715238 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.715253 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.716261 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.716265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.717306 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.717316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.717882 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.717886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.718071 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.718781 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:13.718800 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:13.720377 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:13.722301 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:13.722307 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:13.722323 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:13.722595 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:13.722903 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:13.722906 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:13.722908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:13.723054 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:13.723474 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:13.723482 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:13.723744 (+ 262us) client_negotiation.cc:769] Sending connection context
0808 02:13:13.723940 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 02:13:13.724284 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":131,"threads_started":1}
I20260808 02:13:13.726666 25670 heartbeater.cc:347] Connected to a master server at 127.18.184.190:40527
I20260808 02:13:13.726931 25670 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:13.727551 25670 heartbeater.cc:511] Master 127.18.184.190:40527 requested a full tablet report, sending...
I20260808 02:13:13.728700 25167 ts_manager.cc:194] Registered new tserver with Master: f38f6ab4b3be43f997237f5174640a32 (127.18.184.131:36305)
I20260808 02:13:13.729378 25167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:42435
I20260808 02:13:13.740896 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/test-admin.keytab.
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.762471 19170 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 02:13:13.765642 19170 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/krb5cc' of type FILE
W20260808 02:13:13.765606 25678 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:13.764018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34776 (local address 127.18.184.129:36399)
0808 02:13:13.764343 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.764350 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.764376 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:13:13.764439 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.764445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.764652 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.764793 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.765416 (+ 623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34776: BlockingRecv error: recv got EOF from 127.0.0.1:34776 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":78,"threads_started":1}
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.766834 19170 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.767794 19170 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.129@KRBTEST.COM
I20260808 02:13:13.778393 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.768807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34780 (local address 127.18.184.129:36399)
0808 02:13:13.768953 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.768958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.769291 (+ 333us) server_negotiation.cc:407] Connection header received
0808 02:13:13.769356 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.769358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.769410 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.769500 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.770127 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.771055 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.772005 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.772410 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.772509 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.774977 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:13.774987 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:13.775002 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:13.775029 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:13.776555 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.776993 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.776998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.777000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.777080 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.777390 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.777393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.777394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.777683 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:13.777889 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.778036 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.778224 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 02:13:13.783303 25678 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:13.781719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.18.184.129:36399)
0808 02:13:13.781838 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.781842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.782267 (+ 425us) server_negotiation.cc:407] Connection header received
0808 02:13:13.782391 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.782394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.782437 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.782525 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.783181 (+ 656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34782: BlockingRecv error: recv got EOF from 127.0.0.1:34782 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 02:13:13.783439 19170 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/krb5cc' of type FILE
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.784555 19170 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.785917 19170 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.129@KRBTEST.COM
I20260808 02:13:13.796406 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.787548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.18.184.129:36399)
0808 02:13:13.787700 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.787712 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.787728 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:13:13.787770 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.787773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.787819 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.787910 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.788682 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.789335 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.790353 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.790584 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.790664 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.793111 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:13.793115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:13.793117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:13.793202 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:13.794789 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.795203 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.795206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.795208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.795298 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.795546 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.795549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.795550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.795827 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:13.795979 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.796156 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.796267 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 02:13:13.799336 25678 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:13.798347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34806 (local address 127.18.184.129:36399)
0808 02:13:13.798518 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.798521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.798537 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:13:13.798700 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.798702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.798744 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.798823 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.799221 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34806: BlockingRecv error: recv got EOF from 127.0.0.1:34806 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 02:13:13.799741 19170 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155161841348-19170-0/krb5kdc/krb5cc)
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.800827 19170 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.801774 19170 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.129@KRBTEST.COM
I20260808 02:13:13.810783 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.802613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34822 (local address 127.18.184.129:36399)
0808 02:13:13.802757 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.802760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.802854 (+ 94us) server_negotiation.cc:407] Connection header received
0808 02:13:13.802906 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.802908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.802946 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.803012 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.803477 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.804014 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.804765 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.804966 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.805044 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.807658 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:13.807665 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:13.807668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:13.807705 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:13.809056 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.809534 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.809537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.809539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.809724 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.810018 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.810021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.810022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.810239 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:13.810383 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.810544 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.810653 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:13:13.811941 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 25236
I20260808 02:13:13.821084 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 25383
I20260808 02:13:13.828594 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 25529
I20260808 02:13:13.838254 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 25134
2026-08-08T02:13:13Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2369 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25692](info): set up 2 sockets
Aug 08 02:13:13 dist-test-slave-9gcw krb5kdc[25692](info): commencing operation
krb5kdc: starting...
W20260808 02:13:14.952720 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:13:15 dist-test-slave-9gcw krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.190.keytab.
Entry for principal oryx/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.190.keytab.
I20260808 02:13:15.095997 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:43577
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.190.keytab
--principal=oryx/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:43577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:15.214186 25705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:15.214504 25705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:15.214587 25705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:15.218698 25705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:15.218796 25705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:15.218835 25705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:15.218874 25705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:15.218914 25705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:15.224337 25705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:43577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.190.keytab
--principal=oryx/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:43577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:15.225539 25705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:15.226620 25705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:15.235374 25712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:15.235550 25711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:15.235597 25714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:15.235654 25705 server_base.cc:1034] running on GCE node
I20260808 02:13:15.236465 25705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:15.237659 25705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:15.238832 25705 hybrid_clock.cc:648] HybridClock initialized: now 1786155195238803 us; error 42 us; skew 500 ppm
Aug 08 02:13:15 dist-test-slave-9gcw krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155195, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:15.241844 25705 init.cc:377] Logged in from keytab as oryx/127.18.184.190@KRBTEST.COM (short username oryx)
I20260808 02:13:15.242763 25705 webserver.cc:460] Webserver started at http://127.18.184.190:38951/ using document root <none> and password file <none>
I20260808 02:13:15.243373 25705 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:15.243464 25705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:15.243700 25705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:15.246120 25705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "68f66e73d25e486080cfc178dc1df99f"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-9gcw"
server_key: "b9d54ab1ab781b8bd610a80c3e8e8492"
server_key_iv: "fee6e31c5e235bdfb16a273cd461149f"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.246788 25705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "68f66e73d25e486080cfc178dc1df99f"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-9gcw"
server_key: "b9d54ab1ab781b8bd610a80c3e8e8492"
server_key_iv: "fee6e31c5e235bdfb16a273cd461149f"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.251291 25705 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:13:15.254069 25721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:15.255684 25705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:15.255878 25705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "68f66e73d25e486080cfc178dc1df99f"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-9gcw"
server_key: "b9d54ab1ab781b8bd610a80c3e8e8492"
server_key_iv: "fee6e31c5e235bdfb16a273cd461149f"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.256024 25705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:15.281054 25705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:15.284304 25705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:15.284538 25705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:15.292114 25705 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:43577
I20260808 02:13:15.292428 25776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:43577 every 8 connection(s)
I20260808 02:13:15.293278 25705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:15.293895 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 25705
I20260808 02:13:15.294054 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:15.294353 19170 external_mini_cluster.cc:1428] Setting key 93ff609b815231a1fc3a822614a4aeb8
I20260808 02:13:15.297241 25777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 02:13:15 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
I20260808 02:13:15.303403 25777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f: Bootstrap starting.
I20260808 02:13:15.305984 25777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:15.306805 25777 log.cc:824] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:15.309087 25777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f: No bootstrap required, opened a new log
I20260808 02:13:15.309309 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:15.295728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55942 (local address 127.18.184.190:43577)
0808 02:13:15.296130 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:15.296145 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:15.296182 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:13:15.296845 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:15.296862 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:15.297258 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:15.297651 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:15.298482 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.299489 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.300528 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.301036 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.301252 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:15.303979 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:15.303984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:15.303996 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:15.304034 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:15.305973 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:15.306455 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:15.306461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:15.306465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:15.306579 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:15.306927 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:15.306933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:15.306935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:15.307447 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:15.307648 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:15.307999 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 02:13:15.308409 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":112,"threads_started":1}
I20260808 02:13:15.313236 25777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } }
I20260808 02:13:15.313532 25777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:15.313599 25777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f66e73d25e486080cfc178dc1df99f, State: Initialized, Role: FOLLOWER
I20260808 02:13:15.314173 25777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } }
I20260808 02:13:15.314337 25777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:15.314404 25777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:15.314502 25777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:15.315507 25777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } }
I20260808 02:13:15.315868 25777 leader_election.cc:302] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68f66e73d25e486080cfc178dc1df99f; no voters:
I20260808 02:13:15.316213 25777 leader_election.cc:288] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:15.316382 25782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:15.316664 25782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 1 LEADER]: Becoming Leader. State: Replica: 68f66e73d25e486080cfc178dc1df99f, State: Running, Role: LEADER
I20260808 02:13:15.316960 25782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } }
I20260808 02:13:15.317593 25777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:15.318725 25784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68f66e73d25e486080cfc178dc1df99f. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } }
I20260808 02:13:15.318858 25784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:15.319228 25783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } }
I20260808 02:13:15.319348 25792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:15.319429 25783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:15.323002 25792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:15.328537 25792 catalog_manager.cc:1347] Generated new cluster ID: 43c29d2eb3a54bbc9d7df32a70270782
I20260808 02:13:15.328616 25792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:15.334841 25792 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:15.336057 25792 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:15.348626 25792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f: Generated new TSK 0
I20260808 02:13:15.349758 25792 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab.
Entry for principal oryx/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab.
I20260808 02:13:15.389664 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:43577
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab
--principal=oryx/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:15.534792 25808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:15.535068 25808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:15.535113 25808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:15.539355 25808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:15.539429 25808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:15.539490 25808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:15.544870 25808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab
--principal=oryx/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:43577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:15.546000 25808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:15.547019 25808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:15.557176 25817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:15.557188 25814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:15.557183 25813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:15.557590 25808 server_base.cc:1034] running on GCE node
I20260808 02:13:15.558027 25808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:15.558619 25808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:15.559888 25808 hybrid_clock.cc:648] HybridClock initialized: now 1786155195559863 us; error 49 us; skew 500 ppm
Aug 08 02:13:15 dist-test-slave-9gcw krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155195, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:15.563102 25808 init.cc:377] Logged in from keytab as oryx/127.18.184.129@KRBTEST.COM (short username oryx)
I20260808 02:13:15.564059 25808 webserver.cc:460] Webserver started at http://127.18.184.129:38901/ using document root <none> and password file <none>
I20260808 02:13:15.564700 25808 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:15.564802 25808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:15.565042 25808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:15.567183 25808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "236b94a7ada74cc28c42bf86c2db8575"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-9gcw"
server_key: "935abc9c4e400ae1172c167dbe4ace7f"
server_key_iv: "1307de69a1da51ff4d3908a2d4fd4428"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.567799 25808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "236b94a7ada74cc28c42bf86c2db8575"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-9gcw"
server_key: "935abc9c4e400ae1172c167dbe4ace7f"
server_key_iv: "1307de69a1da51ff4d3908a2d4fd4428"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.571887 25808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 02:13:15.574533 25824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:15.575882 25808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:15.575982 25808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "236b94a7ada74cc28c42bf86c2db8575"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-9gcw"
server_key: "935abc9c4e400ae1172c167dbe4ace7f"
server_key_iv: "1307de69a1da51ff4d3908a2d4fd4428"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.576074 25808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:15.596426 25808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:15.599851 25808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:15.600018 25808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:15.600590 25808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:15.601513 25808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:15.601565 25808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:15.601608 25808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:15.601630 25808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:15.612031 25808 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:33407
I20260808 02:13:15.612051 25945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:33407 every 8 connection(s)
I20260808 02:13:15.613050 25808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:15.618904 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 25808
I20260808 02:13:15.619021 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:15.619454 19170 external_mini_cluster.cc:1428] Setting key b97096b6646a20cb3d063c579460e455
Aug 08 02:13:15 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155195, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
I20260808 02:13:15.629496 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:15.615096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:37435 (local address 127.18.184.190:43577)
0808 02:13:15.615281 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:15.615286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:15.616185 (+ 899us) server_negotiation.cc:407] Connection header received
0808 02:13:15.617648 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:15.617651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:15.617775 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:15.617853 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:15.619304 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.620156 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.621588 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.622074 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.622209 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:15.625004 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:15.625007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:15.625009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:15.625034 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:15.626751 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:15.627470 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:15.627473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:15.627475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:15.627568 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:15.628141 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:15.628145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:15.628146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:15.628345 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:15.628431 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:15.629226 (+ 795us) server_negotiation.cc:299] Negotiation successful
0808 02:13:15.629353 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:13:15.630771 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:15.615532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.129:37435)
0808 02:13:15.616007 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:15.616053 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:15.617447 (+ 1394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:15.618028 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:15.618034 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:15.618505 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:15.619144 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:15.619161 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.620317 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.620321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:15.621460 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:15.621470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.622255 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.622259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:15.622422 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:15.623099 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:15.623117 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:15.624795 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:15.626883 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:15.626892 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:15.626915 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:15.627343 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:15.627771 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:15.627776 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:15.627781 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:15.628012 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:15.628471 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:15.628480 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:15.628863 (+ 383us) client_negotiation.cc:769] Sending connection context
0808 02:13:15.629111 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 02:13:15.629535 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":124,"threads_started":1}
WARNING: no policy specified for oryx/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:13:15.632319 25946 heartbeater.cc:347] Connected to a master server at 127.18.184.190:43577
I20260808 02:13:15.632544 25946 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:15.633035 25946 heartbeater.cc:511] Master 127.18.184.190:43577 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.130@KRBTEST.COM" created.
I20260808 02:13:15.634805 25738 ts_manager.cc:194] Registered new tserver with Master: 236b94a7ada74cc28c42bf86c2db8575 (127.18.184.129:33407)
I20260808 02:13:15.636163 25738 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:37435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab.
Entry for principal oryx/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab.
I20260808 02:13:15.652693 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:43577
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab
--principal=oryx/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:15.773437 25952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:15.773733 25952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:15.773809 25952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:15.778079 25952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:15.778147 25952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:15.778215 25952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:15.784150 25952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab
--principal=oryx/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:43577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:15.785463 25952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:15.786571 25952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:15.798094 25961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:15.798058 25959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:15.798084 25958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:15.798583 25952 server_base.cc:1034] running on GCE node
I20260808 02:13:15.799182 25952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:15.799916 25952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:15.801186 25952 hybrid_clock.cc:648] HybridClock initialized: now 1786155195801113 us; error 42 us; skew 500 ppm
Aug 08 02:13:15 dist-test-slave-9gcw krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155195, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:15.804309 25952 init.cc:377] Logged in from keytab as oryx/127.18.184.130@KRBTEST.COM (short username oryx)
I20260808 02:13:15.805435 25952 webserver.cc:460] Webserver started at http://127.18.184.130:36959/ using document root <none> and password file <none>
I20260808 02:13:15.806069 25952 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:15.806166 25952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:15.806427 25952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:15.808806 25952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "bc88b6ae2fd34d69b4d17083d79641e1"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-9gcw"
server_key: "650147b301d7e0c201d292c0b16f5b16"
server_key_iv: "54169c067fc16188bdaf668c296f2d85"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.809481 25952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "bc88b6ae2fd34d69b4d17083d79641e1"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-9gcw"
server_key: "650147b301d7e0c201d292c0b16f5b16"
server_key_iv: "54169c067fc16188bdaf668c296f2d85"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.813762 25952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:13:15.816501 25969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:15.818073 25952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:15.818228 25952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "bc88b6ae2fd34d69b4d17083d79641e1"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-9gcw"
server_key: "650147b301d7e0c201d292c0b16f5b16"
server_key_iv: "54169c067fc16188bdaf668c296f2d85"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.818354 25952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:15.831100 25952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:15.834802 25952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:15.835038 25952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:15.835780 25952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:15.836766 25952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:15.836839 25952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:15.836920 25952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:15.836963 25952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:15.846911 25952 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:41985
I20260808 02:13:15.846931 26086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:41985 every 8 connection(s)
I20260808 02:13:15.847960 25952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:15.851190 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 25952
I20260808 02:13:15.851317 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:15.851651 19170 external_mini_cluster.cc:1428] Setting key 4f2b6d992bfdcae82bf8b8ea9b45713c
Aug 08 02:13:15 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155195, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:13:15.863857 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:15.850184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:37707 (local address 127.18.184.190:43577)
0808 02:13:15.850358 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:15.850364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:15.851202 (+ 838us) server_negotiation.cc:407] Connection header received
0808 02:13:15.852883 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:15.852887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:15.852941 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:15.853024 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:15.854713 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.855405 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.856621 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.856956 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.857056 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:15.859748 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:15.859751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:15.859753 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:15.859779 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:15.861625 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:15.862251 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:15.862255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:15.862257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:15.862346 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:15.862719 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:15.862723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:15.862725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:15.862924 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:15.863035 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:15.863552 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 02:13:15.863685 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:13:15.864933 26090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:15.850511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.130:37707)
0808 02:13:15.851042 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:15.851087 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:15.852669 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:15.853227 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:15.853237 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:15.853765 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:15.854495 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:15.854518 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.855566 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.855569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:15.856483 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:15.856494 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.857110 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.857113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:15.857363 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:15.858036 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:15.858053 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:15.859548 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:15.861823 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:15.861829 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:15.861843 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:15.862114 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:15.862445 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:15.862448 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:15.862450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:15.862601 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:15.863034 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:15.863043 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:15.863313 (+ 270us) client_negotiation.cc:769] Sending connection context
0808 02:13:15.863509 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 02:13:15.863904 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":171,"threads_started":1}
I20260808 02:13:15.866580 26088 heartbeater.cc:347] Connected to a master server at 127.18.184.190:43577
I20260808 02:13:15.866998 26088 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.131@KRBTEST.COM" created.
I20260808 02:13:15.867717 26088 heartbeater.cc:511] Master 127.18.184.190:43577 requested a full tablet report, sending...
I20260808 02:13:15.869048 25738 ts_manager.cc:194] Registered new tserver with Master: bc88b6ae2fd34d69b4d17083d79641e1 (127.18.184.130:41985)
I20260808 02:13:15.869946 25738 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:37707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab.
Entry for principal oryx/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab.
I20260808 02:13:15.888888 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:43577
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab
--principal=oryx/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:16.006992 26093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:16.007284 26093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:16.007329 26093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:16.011533 26093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:16.011608 26093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:16.011675 26093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:16.017398 26093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab
--principal=oryx/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:43577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:16.018735 26093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:16.019893 26093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:16.030834 26102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:16.030858 26099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:16.030846 26100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:16.031263 26093 server_base.cc:1034] running on GCE node
I20260808 02:13:16.032033 26093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:16.032804 26093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:16.033981 26093 hybrid_clock.cc:648] HybridClock initialized: now 1786155196033951 us; error 51 us; skew 500 ppm
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:16.037272 26093 init.cc:377] Logged in from keytab as oryx/127.18.184.131@KRBTEST.COM (short username oryx)
I20260808 02:13:16.038345 26093 webserver.cc:460] Webserver started at http://127.18.184.131:45121/ using document root <none> and password file <none>
I20260808 02:13:16.038895 26093 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:16.038961 26093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:16.039140 26093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:16.041366 26093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "ce6ceb70486646e3a3e812c025e35998"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "2c8c0495bdaa2afa149f00c84bfd7844"
server_key_iv: "c06dcd3646669045b9e45a0ba8d84d01"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.041931 26093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "ce6ceb70486646e3a3e812c025e35998"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "2c8c0495bdaa2afa149f00c84bfd7844"
server_key_iv: "c06dcd3646669045b9e45a0ba8d84d01"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.046046 26093 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:13:16.048750 26110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:16.050275 26093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:13:16.050423 26093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "ce6ceb70486646e3a3e812c025e35998"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "2c8c0495bdaa2afa149f00c84bfd7844"
server_key_iv: "c06dcd3646669045b9e45a0ba8d84d01"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.050549 26093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:16.065196 26093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:16.068810 26093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:16.069032 26093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:16.069706 26093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:16.070556 26093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:16.070639 26093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:16.070736 26093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:16.070787 26093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:16.081882 26093 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:44559
I20260808 02:13:16.081916 26229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:44559 every 8 connection(s)
I20260808 02:13:16.082958 26093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:16.087476 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 26093
I20260808 02:13:16.087615 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:16.087958 19170 external_mini_cluster.cc:1428] Setting key 06a62ebf978000d03eb52ae261d7526e
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
I20260808 02:13:16.099375 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.085297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:53521 (local address 127.18.184.190:43577)
0808 02:13:16.085482 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.085486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.086404 (+ 918us) server_negotiation.cc:407] Connection header received
0808 02:13:16.088475 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.088478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.088543 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.088632 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.090088 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.090766 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.092040 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.092314 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.092440 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.095256 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.095260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.095262 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.095300 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.096938 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.097600 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.097604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.097606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.097701 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.098080 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.098084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.098086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.098319 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.098470 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.099028 (+ 558us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.099179 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 02:13:16.100250 26233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.085705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.131:53521)
0808 02:13:16.086227 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.086277 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.088216 (+ 1939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.088827 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.088834 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.089396 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.089933 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.089948 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.090910 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.090920 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.091898 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.091908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.092488 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.092492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.092687 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.093421 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.093439 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.095067 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.097077 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.097084 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.097102 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.097477 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.097803 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.097807 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.097809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.097967 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.098469 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.098476 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.098763 (+ 287us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.098966 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.099333 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":130,"threads_started":1}
I20260808 02:13:16.101824 26230 heartbeater.cc:347] Connected to a master server at 127.18.184.190:43577
I20260808 02:13:16.102136 26230 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:16.102823 26230 heartbeater.cc:511] Master 127.18.184.190:43577 requested a full tablet report, sending...
I20260808 02:13:16.103921 25736 ts_manager.cc:194] Registered new tserver with Master: ce6ceb70486646e3a3e812c025e35998 (127.18.184.131:44559)
I20260808 02:13:16.104553 25736 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:53521
I20260808 02:13:16.114763 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.184.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.189.keytab.
Entry for principal oryx/127.18.184.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.189.keytab.
I20260808 02:13:16.149016 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.189:44331
--webserver_interface=127.18.184.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.189.keytab
--principal=oryx/127.18.184.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:43577,127.18.184.189:44331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:16.268815 26237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:16.269130 26237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:16.269259 26237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:16.273622 26237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:16.273701 26237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:16.273723 26237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:16.273742 26237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:16.273758 26237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:16.279623 26237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:43577,127.18.184.189:44331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.189.keytab
--principal=oryx/127.18.184.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.189:44331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.184.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
W20260808 02:13:16.279884 26237 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.184.190:43577,127.18.184.189:44331'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 02:13:16.280866 26237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:16.281921 26237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:16.291165 26246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:16.291293 26237 server_base.cc:1034] running on GCE node
W20260808 02:13:16.291172 26244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:16.291172 26243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:16.292236 26237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:16.293589 26237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:16.294832 26237 hybrid_clock.cc:648] HybridClock initialized: now 1786155196294809 us; error 45 us; skew 500 ppm
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:16.298081 26237 init.cc:377] Logged in from keytab as oryx/127.18.184.189@KRBTEST.COM (short username oryx)
I20260808 02:13:16.299078 26237 webserver.cc:460] Webserver started at http://127.18.184.189:42783/ using document root <none> and password file <none>
I20260808 02:13:16.299656 26237 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:16.299726 26237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:16.299952 26237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:16.302246 26237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data/instance:
uuid: "f9a3d956916147c9ad89c32fc1fe6317"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "7630be968bfff2a9afee595c1ec4a7ba"
server_key_iv: "1ef08ff37c6f454e1cd61fbd05cd5a1b"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.302898 26237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/wal/instance:
uuid: "f9a3d956916147c9ad89c32fc1fe6317"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "7630be968bfff2a9afee595c1ec4a7ba"
server_key_iv: "1ef08ff37c6f454e1cd61fbd05cd5a1b"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.306994 26237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 02:13:16.309732 26253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:16.311276 26237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:16.311468 26237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/wal
uuid: "f9a3d956916147c9ad89c32fc1fe6317"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "7630be968bfff2a9afee595c1ec4a7ba"
server_key_iv: "1ef08ff37c6f454e1cd61fbd05cd5a1b"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.311612 26237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:16.332758 26237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:16.336795 26237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:16.337044 26237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:16.345484 26237 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.189:44331
I20260808 02:13:16.345821 26311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.189:44331 every 8 connection(s)
I20260808 02:13:16.346731 26237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
I20260808 02:13:16.347211 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 26237
I20260808 02:13:16.350737 26312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:16.356177 26312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.184.189@KRBTEST.COM
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.189@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
I20260808 02:13:16.365569 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.349107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57646 (local address 127.18.184.189:44331)
0808 02:13:16.349683 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.349705 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.349758 (+ 53us) server_negotiation.cc:407] Connection header received
0808 02:13:16.350733 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.350761 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.351247 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.351672 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.352475 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.354275 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.355526 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.356299 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.356478 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.359569 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.359575 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.359584 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.359611 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.361958 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.362553 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.362560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.362566 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.362764 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.363142 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.363146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.363148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.363596 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.363849 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.364211 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.364558 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":161,"threads_started":1}
I20260808 02:13:16.368954 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.358345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55950 (local address 127.18.184.190:43577)
0808 02:13:16.358526 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.358531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.358936 (+ 405us) server_negotiation.cc:407] Connection header received
0808 02:13:16.359213 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.359216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.359267 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.359345 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.360241 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.361176 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.362543 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.362972 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.363116 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.365482 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.365487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.365490 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.365522 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.367074 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.367607 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.367611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.367615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.367706 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.368054 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.368056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.368058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.368298 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.368384 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.368732 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.368840 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:13:16.369010 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.358298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:55950)
0808 02:13:16.358796 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.358856 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.359043 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.359439 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.359448 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.359815 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.360043 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.360054 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.361339 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.361343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.362404 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.362415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.363114 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.363120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.363240 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.363807 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.363819 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.365337 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.367214 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.367224 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.367230 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.367484 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.367818 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.367821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.367823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.367942 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.368423 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.368428 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.368533 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.368680 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.368832 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":166,"threads_started":1}
I20260808 02:13:16.370326 26312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.189@KRBTEST.COM for oryx/127.18.184.189@KRBTEST.COM
I20260808 02:13:16.378849 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.370942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:44331 (local address 127.0.0.1:57654)
0808 02:13:16.371087 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.371102 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.371212 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.371445 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.371448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.371601 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.371679 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.371684 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.372779 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.372782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.373337 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.373345 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.373775 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.373778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.373892 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.374356 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.374368 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.375548 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.377309 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.377312 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.377315 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.377513 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.377809 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.377813 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.377815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.377897 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.378293 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.378296 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.378382 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.378494 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.378647 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":1}
I20260808 02:13:16.378849 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.371004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.18.184.189:44331)
0808 02:13:16.371108 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.371114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.371163 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:13:16.371320 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.371323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.371356 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.371434 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.371832 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.372654 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.373447 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.373661 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.373778 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.375677 (+ 1899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.375681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.375690 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.375710 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.377112 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.377640 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.377643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.377644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.377712 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.377988 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.377991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.377992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.378194 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.378305 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.378552 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.378701 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":49}
I20260808 02:13:16.382587 26312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: Bootstrap starting.
I20260808 02:13:16.385008 26312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:16.385828 26312 log.cc:824] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:16.388055 26312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: No bootstrap required, opened a new log
I20260808 02:13:16.391206 26312 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } }
I20260808 02:13:16.391424 26312 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:16.391494 26312 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9a3d956916147c9ad89c32fc1fe6317, State: Initialized, Role: FOLLOWER
I20260808 02:13:16.391937 26312 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } }
I20260808 02:13:16.392481 26319 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } } }
I20260808 02:13:16.392699 26319 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:13:16.393343 26312 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 02:13:16.398634 26331 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:13:16.398733 26331 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:13:16.400959 26237 master_runner.cc:428] Detected that this master f9a3d956916147c9ad89c32fc1fe6317 is joining an existing cluster
I20260808 02:13:16.401027 26237 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.184.189:44331
I20260808 02:13:16.413615 26341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.403942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:44331 (local address 127.0.0.1:57664)
0808 02:13:16.404311 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.404322 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.404438 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.404789 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.404791 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.405016 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.405207 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.405216 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.406008 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.406012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.407172 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.407184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.407818 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.407821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.407899 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.408774 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.408793 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.409332 (+ 539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.411902 (+ 2570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.411905 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.411907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.412147 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.412448 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.412451 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.412452 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.412527 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.412982 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.412985 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.413080 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.413246 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.413397 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":66,"thread_start_us":83,"threads_started":1}
I20260808 02:13:16.413980 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.404035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57664 (local address 127.18.184.189:44331)
0808 02:13:16.404142 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.404146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.404406 (+ 260us) server_negotiation.cc:407] Connection header received
0808 02:13:16.404561 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.404566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.404612 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.404691 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.405328 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.405878 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.407305 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.407655 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.407769 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.409477 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.409481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.409484 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.409509 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.411782 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.412251 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.412254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.412256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.412355 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.412653 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.412660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.412665 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.412889 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.412998 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.413314 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.413474 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":44,"spinlock_wait_cycles":768}
I20260808 02:13:16.414404 26340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.403443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:55952)
0808 02:13:16.403921 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.403936 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.404099 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.404467 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.404474 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.404788 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.404970 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.404979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.406068 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.406071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.407172 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.407184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.408378 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.408391 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.408504 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.409477 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.409488 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.410011 (+ 523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.412244 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.412250 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.412252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.412530 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.412928 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.412934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.412939 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.413037 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.413635 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.413638 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.413959 (+ 321us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.414091 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.414263 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":72,"spinlock_wait_cycles":2816,"thread_start_us":110,"threads_started":1}
I20260808 02:13:16.415309 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.403522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55952 (local address 127.18.184.190:43577)
0808 02:13:16.403710 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.403715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.404012 (+ 297us) server_negotiation.cc:407] Connection header received
0808 02:13:16.404246 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.404250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.404308 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.404401 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.405110 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.405971 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.407887 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.408240 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.408350 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.410211 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.410215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.410218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.410254 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.412034 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.412729 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.412733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.412735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.412820 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.413221 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.413224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.413226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.413471 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.413554 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.415024 (+ 1470us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.415147 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155195, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.190@KRBTEST.COM for oryx/127.18.184.189@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.184.188@KRBTEST.COM; defaulting to no policy
I20260808 02:13:16.431691 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.420340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57672 (local address 127.18.184.189:44331)
0808 02:13:16.421087 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.421094 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.421113 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:13:16.421206 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.421213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.421283 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.421385 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.422148 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.422919 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.424198 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.424576 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.424654 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.427521 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.427525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.427528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.427554 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.429410 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.430101 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.430104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.430106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.430191 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.430550 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.430554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.430555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.430951 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.431047 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.431459 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.431563 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642}
I20260808 02:13:16.431803 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.420311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:44331 (local address 127.0.0.1:57672)
0808 02:13:16.420715 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.420775 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.420894 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.421438 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.421444 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.421875 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.422007 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.422015 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.423061 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.423064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.424059 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.424070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.424816 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.424820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.424930 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.425547 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.425562 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.427353 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.429548 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.429558 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.429564 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.429867 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.430296 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.430300 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.430302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.430401 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.431076 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.431084 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.431238 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.431401 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.431618 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.188@KRBTEST.COM" created.
I20260808 02:13:16.433605 25736 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.184.189:44331: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317"
member_type: NON_VOTER
last_known_addr {
host: "127.18.184.189"
port: 44331
}
attrs {
promote: true
}
}
dest_uuid: "68f66e73d25e486080cfc178dc1df99f"
cas_config_opid_index: -1
I20260808 02:13:16.434396 25736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true } }
I20260808 02:13:16.436548 26346 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f9a3d956916147c9ad89c32fc1fe6317 (127.18.184.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true } } }
I20260808 02:13:16.437408 26284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:16.438508 26346 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.184.189:44331
I20260808 02:13:16.438874 26284 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 FOLLOWER]: Refusing update from remote peer 68f66e73d25e486080cfc178dc1df99f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 02:13:16.439091 26237 master.cc:537] Master@127.18.184.189:44331 shutting down...
I20260808 02:13:16.439582 26346 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:16.439783 26347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true } } }
I20260808 02:13:16.439913 26347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: This master's current role is: LEADER
W20260808 02:13:16.441884 25724 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260808 02:13:16.443681 26237 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:16.444092 26237 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:16.444295 26237 tablet_replica.cc:330] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: stopping tablet replica
I20260808 02:13:16.450021 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.449693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:44331 (local address 127.0.0.1:57686)
0808 02:13:16.449828 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.449891 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 02:13:16.450179 25724 consensus_peers.cc:487] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f -> Peer f9a3d956916147c9ad89c32fc1fe6317 (127.18.184.189:44331): Couldn't send request to peer f9a3d956916147c9ad89c32fc1fe6317. Status: Network error: Client connection negotiation failed: client connection to 127.18.184.189:44331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.188.keytab.
Entry for principal oryx/127.18.184.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.188.keytab.
I20260808 02:13:16.456853 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.188:35335
--webserver_interface=127.18.184.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.188.keytab
--principal=oryx/127.18.184.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:43577,127.18.184.189:44331,127.18.184.188:35335
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:13:16.460624 26237 master.cc:552] Master@127.18.184.189:44331 shutdown complete.
I20260808 02:13:16.460696 26237 master_runner.cc:305] Clearing existing system tablet
I20260808 02:13:16.461001 26237 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:16.461107 26237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:16.462633 26237 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:16.463533 26352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:16.463743 26237 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:16.463877 26237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:16.463914 26237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/wal
uuid: "f9a3d956916147c9ad89c32fc1fe6317"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "7630be968bfff2a9afee595c1ec4a7ba"
server_key_iv: "1ef08ff37c6f454e1cd61fbd05cd5a1b"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.465813 26237 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:16.467805 26237 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 02:13:16.467922 26237 log.cc:1196] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 02:13:16.468165 26237 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: Deleting consensus metadata
I20260808 02:13:16.468429 26237 master_runner.cc:315] Copying system tablet from 127.18.184.190:43577
I20260808 02:13:16.469780 26237 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: tablet copy: Beginning tablet copy session from remote peer at address 127.18.184.190:43577
I20260808 02:13:16.480950 26358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.470413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:55962)
0808 02:13:16.473370 (+ 2957us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.473384 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.473553 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.473913 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.473916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.474125 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.474249 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.474262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.475046 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.475051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.475653 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.475661 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.476171 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.476175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.476281 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.476868 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.476880 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.477327 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.479265 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.479269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.479271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.479483 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.479803 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.479806 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.479808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.479904 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.480358 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.480363 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.480469 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.480573 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.480730 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2883,"thread_start_us":101,"threads_started":1}
I20260808 02:13:16.480939 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.470480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55962 (local address 127.18.184.190:43577)
0808 02:13:16.470650 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.470655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.473492 (+ 2837us) server_negotiation.cc:407] Connection header received
0808 02:13:16.473672 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.473675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.473734 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.473879 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.474385 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.474929 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.475793 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.476037 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.476148 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.477473 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.477476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.477479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.477508 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.479138 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.479592 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.479595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.479597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.479689 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.480007 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.480011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.480012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.480242 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.480355 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.480648 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.480776 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 02:13:16.481952 25760 tablet_copy_service.cc:140] P 68f66e73d25e486080cfc178dc1df99f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f9a3d956916147c9ad89c32fc1fe6317 ({username='oryx', principal='oryx/127.18.184.189@KRBTEST.COM'} at 127.0.0.1:55962)
I20260808 02:13:16.482185 25760 tablet_copy_service.cc:161] P 68f66e73d25e486080cfc178dc1df99f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f9a3d956916147c9ad89c32fc1fe6317 at {username='oryx', principal='oryx/127.18.184.189@KRBTEST.COM'} at 127.0.0.1:55962: session id = f9a3d956916147c9ad89c32fc1fe6317-00000000000000000000000000000000
I20260808 02:13:16.483726 25760 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 02:13:16.485370 26237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 02:13:16.487520 26237 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: tablet copy: Starting download of 0 data blocks...
I20260808 02:13:16.487742 26237 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: tablet copy: Starting download of 1 WAL segments...
I20260808 02:13:16.489459 26237 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 02:13:16.491048 25760 tablet_copy_service.cc:342] P 68f66e73d25e486080cfc178dc1df99f: Request end of tablet copy session f9a3d956916147c9ad89c32fc1fe6317-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.184.189@KRBTEST.COM'} at 127.0.0.1:55962
I20260808 02:13:16.491209 25760 tablet_copy_service.cc:433] P 68f66e73d25e486080cfc178dc1df99f: ending tablet copy session f9a3d956916147c9ad89c32fc1fe6317-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f9a3d956916147c9ad89c32fc1fe6317
I20260808 02:13:16.498628 26237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:16.500349 26364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:16.500350 26360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:16.500910 26361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:16.501066 26237 server_base.cc:1034] running on GCE node
I20260808 02:13:16.501324 26237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:16.501657 26237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:16.502797 26237 hybrid_clock.cc:648] HybridClock initialized: now 1786155196502783 us; error 26 us; skew 500 ppm
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:16.504590 26237 init.cc:377] Logged in from keytab as oryx/127.18.184.189@KRBTEST.COM (short username oryx)
I20260808 02:13:16.505194 26237 webserver.cc:460] Webserver started at http://127.18.184.189:38041/ using document root <none> and password file <none>
I20260808 02:13:16.505430 26237 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:16.505493 26237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:16.506832 26237 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:16.507901 26371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:16.508163 26237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:16.508271 26237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/wal
uuid: "f9a3d956916147c9ad89c32fc1fe6317"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "7630be968bfff2a9afee595c1ec4a7ba"
server_key_iv: "1ef08ff37c6f454e1cd61fbd05cd5a1b"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.508387 26237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:16.520695 26237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:16.523008 26237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:16.528155 26237 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.189:44331
I20260808 02:13:16.528891 26237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
I20260808 02:13:16.529346 26429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.189:44331 every 8 connection(s)
I20260808 02:13:16.530619 26430 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:13:16.531569 26430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: Bootstrap starting.
I20260808 02:13:16.540307 26430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:13:16.540701 26430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: Bootstrap complete.
I20260808 02:13:16.541333 26430 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true } }
I20260808 02:13:16.541473 26430 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f9a3d956916147c9ad89c32fc1fe6317, State: Initialized, Role: LEARNER
I20260808 02:13:16.541592 26430 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true } }
I20260808 02:13:16.541960 26433 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true } } }
I20260808 02:13:16.542142 26433 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:13:16.542200 26430 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:16.545467 26446 catalog_manager.cc:1259] Loaded cluster ID: 43c29d2eb3a54bbc9d7df32a70270782
I20260808 02:13:16.545576 26446 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: loading cluster ID for follower catalog manager: success
I20260808 02:13:16.547260 26446 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: acquiring CA information for follower catalog manager: success
I20260808 02:13:16.548467 26446 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 02:13:16.611279 26350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:16.611565 26350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:16.611642 26350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:16.616103 26350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:16.616176 26350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:16.616197 26350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:16.616214 26350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:16.616230 26350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:16.622311 26350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:43577,127.18.184.189:44331,127.18.184.188:35335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.188.keytab
--principal=oryx/127.18.184.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.188:35335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.184.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:16.623574 26350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:16.624614 26350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:16.634332 26453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:16.634275 26455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:16.634320 26451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:16.634680 26350 server_base.cc:1034] running on GCE node
I20260808 02:13:16.635370 26350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:16.636616 26350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:16.637847 26350 hybrid_clock.cc:648] HybridClock initialized: now 1786155196637829 us; error 33 us; skew 500 ppm
I20260808 02:13:16.639147 25946 heartbeater.cc:503] Master 127.18.184.190:43577 was elected leader, sending a full tablet report...
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:16.641310 26350 init.cc:377] Logged in from keytab as oryx/127.18.184.188@KRBTEST.COM (short username oryx)
I20260808 02:13:16.642355 26350 webserver.cc:460] Webserver started at http://127.18.184.188:41479/ using document root <none> and password file <none>
I20260808 02:13:16.642956 26350 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:16.643025 26350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:16.643326 26350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:16.645643 26350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data/instance:
uuid: "6728af60b1d24132b3fe6e66c733de88"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "5905067cecf23e704a491307d8932866"
server_key_iv: "22163bad6be0a27319dcfe615224248e"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.646202 26350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/wal/instance:
uuid: "6728af60b1d24132b3fe6e66c733de88"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "5905067cecf23e704a491307d8932866"
server_key_iv: "22163bad6be0a27319dcfe615224248e"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.650348 26350 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:13:16.652854 26463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:16.654505 26350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:13:16.654672 26350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/wal
uuid: "6728af60b1d24132b3fe6e66c733de88"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "5905067cecf23e704a491307d8932866"
server_key_iv: "22163bad6be0a27319dcfe615224248e"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.654861 26350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:16.667152 26350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:16.670423 26350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:16.670648 26350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:16.678257 26350 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.188:35335
I20260808 02:13:16.678555 26522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.188:35335 every 8 connection(s)
I20260808 02:13:16.679498 26350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
I20260808 02:13:16.682785 26524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:16.686143 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 26350
I20260808 02:13:16.686867 26524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.188@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.184.188@KRBTEST.COM
I20260808 02:13:16.699595 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.688896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55974 (local address 127.18.184.190:43577)
0808 02:13:16.689037 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.689042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.689321 (+ 279us) server_negotiation.cc:407] Connection header received
0808 02:13:16.689706 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.689709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.689772 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.689862 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.690685 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.691210 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.692232 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.692455 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.692563 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.695438 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.695442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.695445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.695477 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.697428 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.698022 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.698026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.698028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.698118 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.698486 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.698490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.698492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.698708 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.698823 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.699291 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.699422 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:13:16.700592 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.687765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42118 (local address 127.18.184.188:35335)
0808 02:13:16.688211 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.688231 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.688289 (+ 58us) server_negotiation.cc:407] Connection header received
0808 02:13:16.689210 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.689242 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.689540 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.689769 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.690619 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.691477 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.692314 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.692685 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.692808 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.696017 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.696023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.696035 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.696056 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.697790 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.698252 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.698265 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.698266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.698372 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.698670 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.698674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.698676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.699062 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.699290 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.699502 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.699653 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":141,"threads_started":1}
I20260808 02:13:16.701007 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.688639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:55974)
0808 02:13:16.689124 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.689228 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.689522 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.689915 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.689920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.690282 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.690530 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.690543 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.691348 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.691351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.692093 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.692101 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.692620 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.692624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.692772 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.693444 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.693469 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.695246 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.697551 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.697562 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.697579 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.697855 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.698226 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.698229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.698231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.698387 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.698823 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.698832 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.699072 (+ 240us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.699250 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.699586 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"spinlock_wait_cycles":35456,"thread_start_us":169,"threads_started":1}
I20260808 02:13:16.702165 26524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.188@KRBTEST.COM for oryx/127.18.184.189@KRBTEST.COM
I20260808 02:13:16.711585 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.702711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:44331 (local address 127.0.0.1:57688)
0808 02:13:16.702935 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.702947 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.703011 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.703428 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.703432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.703594 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.703695 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.703700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.704617 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.704622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.705249 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.705258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.705733 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.705736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.705833 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.706288 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.706299 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.707588 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.709851 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.709855 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.709858 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.710090 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.710409 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.710413 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.710415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.710506 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.710973 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.710976 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.711091 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.711204 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.711369 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260808 02:13:16.711587 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.702935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57688 (local address 127.18.184.189:44331)
0808 02:13:16.703223 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.703226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.703238 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:13:16.703274 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.703276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.703313 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.703413 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.703815 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.704502 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.705379 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.705608 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.705721 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.707821 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.707824 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.707827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.707850 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.709699 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.710203 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.710208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.710210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.710303 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.710606 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.710611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.710614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.710827 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.710988 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.711276 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.711410 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":107,"threads_started":1}
I20260808 02:13:16.712751 26524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.188@KRBTEST.COM for oryx/127.18.184.188@KRBTEST.COM
I20260808 02:13:16.721194 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.713597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42126 (local address 127.18.184.188:35335)
0808 02:13:16.713680 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.713683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.713744 (+ 61us) server_negotiation.cc:407] Connection header received
0808 02:13:16.713850 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.713852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.713888 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.713954 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.714337 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.714846 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.715584 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.715819 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.715901 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.717924 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.717927 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.717929 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.717949 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.719539 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.719967 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.719971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.719973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.720048 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.720320 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.720325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.720327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.720517 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.720625 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.720860 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.721017 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":28}
I20260808 02:13:16.721194 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.713524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35335 (local address 127.0.0.1:42126)
0808 02:13:16.713659 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.713674 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.713766 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.713984 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.713986 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.714124 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.714200 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.714204 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.714944 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.714946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.715484 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.715492 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.715922 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.715924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.715983 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.716432 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.716443 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.717780 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.719646 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.719652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.719655 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.719857 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.720146 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.720148 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.720150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.720232 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.720613 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.720616 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.720697 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.720814 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.720960 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 02:13:16.724958 26524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: Bootstrap starting.
I20260808 02:13:16.727464 26524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:16.728264 26524 log.cc:824] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:16.730791 26524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: No bootstrap required, opened a new log
I20260808 02:13:16.733948 26524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } }
I20260808 02:13:16.734186 26524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:16.734270 26524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6728af60b1d24132b3fe6e66c733de88, State: Initialized, Role: FOLLOWER
I20260808 02:13:16.734727 26524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } }
I20260808 02:13:16.735257 26530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } } }
I20260808 02:13:16.735394 26530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:13:16.736037 26524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 02:13:16.741130 26541 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:13:16.741263 26541 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:13:16.745059 26350 master_runner.cc:428] Detected that this master 6728af60b1d24132b3fe6e66c733de88 is joining an existing cluster
I20260808 02:13:16.745157 26350 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.184.188:35335
I20260808 02:13:16.754299 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 25808
I20260808 02:13:16.764638 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.748098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57692 (local address 127.18.184.189:44331)
0808 02:13:16.748240 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.748244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.748973 (+ 729us) server_negotiation.cc:407] Connection header received
0808 02:13:16.749147 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.749150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.749229 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.749283 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.749857 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.750663 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.751804 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.752178 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.752278 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.753753 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.753758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.753761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.753790 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.762465 (+ 8675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.763121 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.763125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.763127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.763228 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.763574 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.763578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.763579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.763841 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.763945 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.764314 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.764463 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:13:16.764770 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.748342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:44331 (local address 127.0.0.1:57692)
0808 02:13:16.748874 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.748888 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.749034 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.749419 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.749422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.749614 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.749727 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.749733 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.750809 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.750813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.751664 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.751673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.752452 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.752456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.752561 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.753009 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.753023 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.753591 (+ 568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.762647 (+ 9056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.762655 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.762660 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.762991 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.763334 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.763340 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.763345 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.763484 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.763950 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.763956 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.764114 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.764243 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.764446 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":74,"threads_started":1}
I20260808 02:13:16.769721 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:33407
--local_ip_for_outbound_sockets=127.18.184.129
--tserver_master_addrs=127.18.184.190:43577
--webserver_port=38901
--webserver_interface=127.18.184.129
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab
--principal=oryx/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.184.190:43577,127.18.184.189:44331
--tserver_master_addrs=127.18.184.190:43577,127.18.184.189:44331,127.18.184.188:35335 with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:13:16.774763 26549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.747514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:55978)
0808 02:13:16.747893 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.747910 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.748079 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.748495 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.748501 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.748796 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.748954 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.748964 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.750031 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.750034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.751036 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.751043 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.751761 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.751765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.751850 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.752442 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.752456 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.769592 (+ 17136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.772477 (+ 2885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.772483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.772488 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.772721 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.773098 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.773103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.773106 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.773242 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.773976 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.773979 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.774093 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.774249 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.774447 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":67,"thread_start_us":118,"threads_started":1}
I20260808 02:13:16.775183 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.747969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42142 (local address 127.18.184.188:35335)
0808 02:13:16.748070 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.748075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.748283 (+ 208us) server_negotiation.cc:407] Connection header received
0808 02:13:16.748431 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.748434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.748470 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.748515 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.749173 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.749927 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.751683 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.752004 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.752080 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.770442 (+ 18362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.770446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.770448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.770468 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.772459 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.773008 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.773011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.773013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.773089 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.773404 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.773407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.773409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.773627 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.773709 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.774916 (+ 1207us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.775035 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 02:13:16.775313 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.747477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55978 (local address 127.18.184.190:43577)
0808 02:13:16.747640 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.747647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.747981 (+ 334us) server_negotiation.cc:407] Connection header received
0808 02:13:16.748210 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.748214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.748271 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.748331 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.749095 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.749914 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.751173 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.751515 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.751608 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.769823 (+ 18215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.769828 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.769830 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.769864 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.772329 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.772841 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.772847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.772850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.772973 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.773376 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.773382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.773386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.773805 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.773915 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.775034 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.775163 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:13:16.775708 26548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.747520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35335 (local address 127.0.0.1:42142)
0808 02:13:16.748206 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.748218 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.748337 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.748635 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.748637 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.748902 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.749029 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.749036 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.750050 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.750053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.751536 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.751548 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.752223 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.752227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.752315 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.769763 (+ 17448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.769779 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.770284 (+ 505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.772584 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.772587 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.772590 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.772892 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.773214 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.773217 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.773219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.773311 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.774624 (+ 1313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.774633 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.774740 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.775387 (+ 647us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.775528 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":64,"spinlock_wait_cycles":23808,"thread_start_us":150,"threads_started":1}
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155195, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.190@KRBTEST.COM for oryx/127.18.184.188@KRBTEST.COM
I20260808 02:13:16.790488 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.780008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35335 (local address 127.0.0.1:42152)
0808 02:13:16.780157 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.780173 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.780258 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.780672 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.780675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.780882 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.781010 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.781017 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.782244 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.782247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.783135 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.783146 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.783830 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.783834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.783940 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.784516 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.784532 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.786374 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.788671 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.788675 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.788678 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.788939 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.789303 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.789307 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.789309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.789401 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.789889 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.789893 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.790001 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.790122 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.790292 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 02:13:16.791338 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.780219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42152 (local address 127.18.184.188:35335)
0808 02:13:16.780359 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.780366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.780381 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:13:16.780460 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.780465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.780534 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.780650 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.781250 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.782110 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.783273 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.783691 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.783826 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.786553 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.786557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.786560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.786589 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.788517 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.789062 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.789066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.789068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.789193 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.789499 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.789502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.789504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.789746 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.789836 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.791052 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.791183 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:13:16.792836 25736 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.184.188:35335: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6728af60b1d24132b3fe6e66c733de88"
member_type: NON_VOTER
last_known_addr {
host: "127.18.184.188"
port: 35335
}
attrs {
promote: true
}
}
dest_uuid: "68f66e73d25e486080cfc178dc1df99f"
cas_config_opid_index: 5
I20260808 02:13:16.793640 25736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } }
I20260808 02:13:16.794872 26346 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6728af60b1d24132b3fe6e66c733de88 (127.18.184.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:16.796470 26346 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.184.188:35335
I20260808 02:13:16.796651 26346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:16.796763 26346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:16.798265 26495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:16.800156 26495 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 FOLLOWER]: Refusing update from remote peer 68f66e73d25e486080cfc178dc1df99f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 02:13:16.800725 26350 master.cc:537] Master@127.18.184.188:35335 shutting down...
I20260808 02:13:16.800902 26347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:13:16.802858 25722 consensus_peers.cc:487] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f -> Peer 6728af60b1d24132b3fe6e66c733de88 (127.18.184.188:35335): Couldn't send request to peer 6728af60b1d24132b3fe6e66c733de88. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 02:13:16.806442 26350 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:16.806740 26350 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:16.806842 26350 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: stopping tablet replica
I20260808 02:13:16.810537 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.794980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:44331 (local address 127.0.0.1:57702)
0808 02:13:16.802784 (+ 7804us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.802801 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.805430 (+ 2629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.806166 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.806169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.806189 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:16.806285 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.806290 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.807518 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.807521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.808823 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.808836 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.810050 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.810053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.810194 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.810246 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.810321 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.810395 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7712}
I20260808 02:13:16.811007 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.795248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57702 (local address 127.18.184.189:44331)
0808 02:13:16.802414 (+ 7166us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.802420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.802882 (+ 462us) server_negotiation.cc:407] Connection header received
0808 02:13:16.805601 (+ 2719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.805605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.805772 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.805852 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:16.806417 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.807236 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.808972 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.809703 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.809807 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.809946 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.810812 (+ 866us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.810894 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7035}
I20260808 02:13:16.811940 26402 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 LEARNER]: Refusing update from remote peer 68f66e73d25e486080cfc178dc1df99f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 02:13:16.812391 26347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 02:13:16.814399 26402 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6728af60b1d24132b3fe6e66c733de88 (127.18.184.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:16.816561 26433 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68f66e73d25e486080cfc178dc1df99f. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:16.816680 26433 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:13:16.816753 26433 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: NON_VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: true } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:16.816830 26433 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:13:16.817216 26347 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f: attempting to promote NON_VOTER f9a3d956916147c9ad89c32fc1fe6317 to VOTER
I20260808 02:13:16.817749 26347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } }
I20260808 02:13:16.819185 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.818792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35335 (local address 127.0.0.1:42164)
0808 02:13:16.819039 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.819102 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260808 02:13:16.819419 26402 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 LEARNER]: Refusing update from remote peer 68f66e73d25e486080cfc178dc1df99f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 02:13:16.819856 26347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 02:13:16.820111 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.819871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35335 (local address 127.0.0.1:42180)
0808 02:13:16.820002 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.820046 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260808 02:13:16.820286 25722 consensus_peers.cc:487] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f -> Peer 6728af60b1d24132b3fe6e66c733de88 (127.18.184.188:35335): Couldn't send request to peer 6728af60b1d24132b3fe6e66c733de88. Status: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35335: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 02:13:16.822530 26350 master.cc:552] Master@127.18.184.188:35335 shutdown complete.
I20260808 02:13:16.822589 26350 master_runner.cc:305] Clearing existing system tablet
I20260808 02:13:16.822849 26350 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:16.822912 26350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:16.824199 26347 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f9a3d956916147c9ad89c32fc1fe6317 (127.18.184.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:16.825337 26350 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:16.826090 26347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:16.826212 26347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:16.826354 26347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:16.826423 26347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:16.826961 26402 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f9a3d956916147c9ad89c32fc1fe6317 (127.18.184.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:16.828770 26553 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:16.828899 26553 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:13:16.829095 26559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:16.829442 26350 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:16.829646 26350 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260808 02:13:16.829713 26350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/wal
uuid: "6728af60b1d24132b3fe6e66c733de88"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "5905067cecf23e704a491307d8932866"
server_key_iv: "22163bad6be0a27319dcfe615224248e"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.831935 26350 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:16.834589 26350 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 02:13:16.834791 26350 log.cc:1196] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 02:13:16.835134 26350 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: Deleting consensus metadata
I20260808 02:13:16.835387 26350 master_runner.cc:315] Copying system tablet from 127.18.184.190:43577
I20260808 02:13:16.836661 26350 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: tablet copy: Beginning tablet copy session from remote peer at address 127.18.184.190:43577
I20260808 02:13:16.850721 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.841658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55986 (local address 127.18.184.190:43577)
0808 02:13:16.841811 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:16.841816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:16.842088 (+ 272us) server_negotiation.cc:407] Connection header received
0808 02:13:16.842209 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:16.842212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:16.842268 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:16.842380 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:16.842927 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.843768 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.844967 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.845297 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.845408 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.846917 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:16.846921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:16.846924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:16.846952 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:16.848599 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.849096 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.849100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.849102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.849243 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:16.849667 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:16.849670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:16.849672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:16.849954 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:16.850044 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:16.850423 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 02:13:16.850552 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:13:16.851578 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:16.841699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:55986)
0808 02:13:16.841975 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:16.841990 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:16.842116 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:16.842411 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:16.842414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:16.842639 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:16.842783 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.842794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.843894 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.843897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.844843 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:16.844854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:16.845477 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:16.845482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:16.845589 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:16.846156 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:16.846221 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:16.846743 (+ 522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:16.848727 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.848730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.848732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.848983 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.849454 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:16.849457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:16.849460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:16.849555 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:16.850121 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:16.850124 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:16.850225 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 02:13:16.851203 (+ 978us) client_negotiation.cc:240] Negotiation successful
0808 02:13:16.851381 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":122,"threads_started":1}
I20260808 02:13:16.852344 25760 tablet_copy_service.cc:140] P 68f66e73d25e486080cfc178dc1df99f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6728af60b1d24132b3fe6e66c733de88 ({username='oryx', principal='oryx/127.18.184.188@KRBTEST.COM'} at 127.0.0.1:55986)
I20260808 02:13:16.852464 25760 tablet_copy_service.cc:161] P 68f66e73d25e486080cfc178dc1df99f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6728af60b1d24132b3fe6e66c733de88 at {username='oryx', principal='oryx/127.18.184.188@KRBTEST.COM'} at 127.0.0.1:55986: session id = 6728af60b1d24132b3fe6e66c733de88-00000000000000000000000000000000
I20260808 02:13:16.853696 25760 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 02:13:16.855017 26350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 02:13:16.857520 26350 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: tablet copy: Starting download of 0 data blocks...
I20260808 02:13:16.857733 26350 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: tablet copy: Starting download of 1 WAL segments...
I20260808 02:13:16.859344 26350 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 02:13:16.861555 25760 tablet_copy_service.cc:342] P 68f66e73d25e486080cfc178dc1df99f: Request end of tablet copy session 6728af60b1d24132b3fe6e66c733de88-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.184.188@KRBTEST.COM'} at 127.0.0.1:55986
I20260808 02:13:16.861649 25760 tablet_copy_service.cc:433] P 68f66e73d25e486080cfc178dc1df99f: ending tablet copy session 6728af60b1d24132b3fe6e66c733de88-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6728af60b1d24132b3fe6e66c733de88
I20260808 02:13:16.868810 26350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:16.870713 26572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:16.870986 26569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:16.871132 26350 server_base.cc:1034] running on GCE node
I20260808 02:13:16.872409 26088 heartbeater.cc:503] Master 127.18.184.190:43577 was elected leader, sending a full tablet report...
W20260808 02:13:16.873268 26567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:16.873761 26350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:16.874100 26350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:16.875267 26350 hybrid_clock.cc:648] HybridClock initialized: now 1786155196875254 us; error 31 us; skew 500 ppm
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:16.877036 26350 init.cc:377] Logged in from keytab as oryx/127.18.184.188@KRBTEST.COM (short username oryx)
I20260808 02:13:16.877616 26350 webserver.cc:460] Webserver started at http://127.18.184.188:42381/ using document root <none> and password file <none>
I20260808 02:13:16.877838 26350 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:16.877895 26350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:16.879413 26350 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:13:16.880677 26579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:16.881042 26350 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:16.881177 26350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/wal
uuid: "6728af60b1d24132b3fe6e66c733de88"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "5905067cecf23e704a491307d8932866"
server_key_iv: "22163bad6be0a27319dcfe615224248e"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.881302 26350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:16.888033 26350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:16.890520 26350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:16.895790 26350 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.188:35335
I20260808 02:13:16.895825 26637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.188:35335 every 8 connection(s)
I20260808 02:13:16.896420 26350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
I20260808 02:13:16.898445 26638 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:13:16.899667 26638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: Bootstrap starting.
I20260808 02:13:16.912650 26638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:13:16.913046 26638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: Bootstrap complete.
I20260808 02:13:16.913740 26638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } }
I20260808 02:13:16.913908 26638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6728af60b1d24132b3fe6e66c733de88, State: Initialized, Role: LEARNER
I20260808 02:13:16.914036 26638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } }
I20260808 02:13:16.914449 26641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:16.914631 26638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:16.914649 26641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:13:16.918860 26652 catalog_manager.cc:1259] Loaded cluster ID: 43c29d2eb3a54bbc9d7df32a70270782
I20260808 02:13:16.919070 26652 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: loading cluster ID for follower catalog manager: success
I20260808 02:13:16.921330 26652 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: acquiring CA information for follower catalog manager: success
I20260808 02:13:16.922842 26652 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 02:13:16.924893 26551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:16.925113 26551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:16.925238 26551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:16.930828 26551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:16.930927 26551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:16.931046 26551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:16.936745 26551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab
--principal=oryx/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:33407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=38901
--tserver_master_addrs=127.18.184.190:43577,127.18.184.189:44331,127.18.184.188:35335
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:16.938005 26551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:16.939178 26551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:16.949692 26658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:16.949676 26659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:16.949962 26551 server_base.cc:1034] running on GCE node
W20260808 02:13:16.949676 26661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:16.950618 26551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:16.951231 26551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:16.952392 26551 hybrid_clock.cc:648] HybridClock initialized: now 1786155196952363 us; error 45 us; skew 500 ppm
Aug 08 02:13:16 dist-test-slave-9gcw krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:16.955428 26551 init.cc:377] Logged in from keytab as oryx/127.18.184.129@KRBTEST.COM (short username oryx)
I20260808 02:13:16.956413 26551 webserver.cc:460] Webserver started at http://127.18.184.129:38901/ using document root <none> and password file <none>
I20260808 02:13:16.957032 26551 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:16.957147 26551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:16.961655 26551 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:13:16.964197 26668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:16.965703 26551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:16.965857 26551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "236b94a7ada74cc28c42bf86c2db8575"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-9gcw"
server_key: "935abc9c4e400ae1172c167dbe4ace7f"
server_key_iv: "1307de69a1da51ff4d3908a2d4fd4428"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.966455 26551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:16.994011 26551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:16.997617 26551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:16.997843 26551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:16.998497 26551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:16.999421 26551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:16.999470 26551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:16.999544 26551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:16.999573 26551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:17.010092 26551 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:33407
I20260808 02:13:17.010221 26792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:33407 every 8 connection(s)
I20260808 02:13:17.011224 26551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:17.012154 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 26551
I20260808 02:13:17.012387 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 25952
I20260808 02:13:17.027544 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:41985
--local_ip_for_outbound_sockets=127.18.184.130
--tserver_master_addrs=127.18.184.190:43577
--webserver_port=36959
--webserver_interface=127.18.184.130
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab
--principal=oryx/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.184.190:43577,127.18.184.189:44331
--tserver_master_addrs=127.18.184.190:43577,127.18.184.189:44331,127.18.184.188:35335 with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:13:17 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.189@KRBTEST.COM
Aug 08 02:13:17 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
Aug 08 02:13:17 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.188@KRBTEST.COM
I20260808 02:13:17.038580 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.019677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:34027 (local address 127.18.184.190:43577)
0808 02:13:17.019864 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.019869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.020623 (+ 754us) server_negotiation.cc:407] Connection header received
0808 02:13:17.022022 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.022026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.022083 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.022140 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.024568 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.025453 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.027555 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.027918 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.027998 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.032699 (+ 4701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.032708 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.032715 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.032762 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.035019 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.036213 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.036218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.036219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.036323 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.036857 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.036863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.036866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.037198 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.037295 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.038182 (+ 887us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.038342 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:13:17.038789 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.016807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:42559 (local address 127.18.184.188:35335)
0808 02:13:17.017591 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.017608 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.020642 (+ 3034us) server_negotiation.cc:407] Connection header received
0808 02:13:17.022044 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.022051 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.022130 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.022217 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.024597 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.026032 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.028307 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.028775 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.028885 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.032999 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.033006 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.033012 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.033051 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.035199 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.036207 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.036213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.036217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.036325 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.036992 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.037013 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.037016 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.037650 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.037846 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.038360 (+ 514us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.038525 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":85,"threads_started":1}
I20260808 02:13:17.039800 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.020920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:40797 (local address 127.18.184.189:44331)
0808 02:13:17.021040 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.021044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.021417 (+ 373us) server_negotiation.cc:407] Connection header received
0808 02:13:17.022022 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.022026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.022086 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.022140 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.024568 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.025464 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.027568 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.027918 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.027998 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.032168 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.032173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.032176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.032211 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.035275 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.036213 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.036217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.036219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.036323 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.038120 (+ 1797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.038125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.038126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.038382 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.038465 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.039480 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.039621 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:13:17.040555 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.020064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.129:34027)
0808 02:13:17.020446 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.020521 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.021759 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.022622 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.022632 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.023396 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:17.024370 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.024392 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.025662 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.025665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.027404 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.027415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.028231 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.028235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.028514 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.029952 (+ 1438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:17.029970 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:17.032387 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:17.035383 (+ 2996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.035392 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.035414 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.036069 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.036458 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.036462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.036464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.036704 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.037403 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:17.037414 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:17.037857 (+ 443us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.038827 (+ 970us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.039302 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":382,"spinlock_wait_cycles":64896,"thread_start_us":137,"threads_started":1}
I20260808 02:13:17.040876 26797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.015495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35335 (local address 127.18.184.129:42559)
0808 02:13:17.020447 (+ 4952us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.020522 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.021761 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.022625 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.022635 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.023388 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:17.024385 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.024402 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.026551 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.026558 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.028129 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.028145 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.029743 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.029747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.029849 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.030437 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:17.030459 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:17.032813 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:17.035383 (+ 2570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.035393 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.035413 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.036064 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.036463 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.036468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.036470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.036761 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.037989 (+ 1228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:17.037995 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:17.038123 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.039017 (+ 894us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.039412 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4676,"mutex_wait_us":296,"spinlock_wait_cycles":39168,"thread_start_us":191,"threads_started":1}
I20260808 02:13:17.040570 26801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.020524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:44331 (local address 127.18.184.129:40797)
0808 02:13:17.021302 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.021317 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.021759 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.022622 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.022632 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.023396 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:17.024370 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.024392 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.025662 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.025665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.027404 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.027415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.028232 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.028235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.028513 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.029509 (+ 996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:17.029538 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:17.031913 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:17.035428 (+ 3515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.035431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.035434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.036069 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.036458 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.036462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.036464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.037962 (+ 1498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.038553 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:17.038556 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:17.039246 (+ 690us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.040212 (+ 966us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.040371 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":401,"spinlock_wait_cycles":23040,"thread_start_us":77,"threads_started":1}
I20260808 02:13:17.042106 26795 heartbeater.cc:347] Connected to a master server at 127.18.184.189:44331
I20260808 02:13:17.042290 26794 heartbeater.cc:347] Connected to a master server at 127.18.184.190:43577
I20260808 02:13:17.042372 26795 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:17.042372 26794 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:17.043062 26793 heartbeater.cc:347] Connected to a master server at 127.18.184.188:35335
I20260808 02:13:17.043068 26794 heartbeater.cc:511] Master 127.18.184.190:43577 requested a full tablet report, sending...
I20260808 02:13:17.043071 26795 heartbeater.cc:511] Master 127.18.184.189:44331 requested a full tablet report, sending...
I20260808 02:13:17.043193 26793 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:17.043381 26793 heartbeater.cc:511] Master 127.18.184.188:35335 requested a full tablet report, sending...
I20260808 02:13:17.044629 25736 ts_manager.cc:194] Re-registered known tserver with Master: 236b94a7ada74cc28c42bf86c2db8575 (127.18.184.129:33407)
I20260808 02:13:17.044795 26588 ts_manager.cc:194] Registered new tserver with Master: 236b94a7ada74cc28c42bf86c2db8575 (127.18.184.129:33407)
I20260808 02:13:17.045471 25736 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:34027
I20260808 02:13:17.046057 26392 ts_manager.cc:194] Registered new tserver with Master: 236b94a7ada74cc28c42bf86c2db8575 (127.18.184.129:33407)
I20260808 02:13:17.106757 26230 heartbeater.cc:503] Master 127.18.184.190:43577 was elected leader, sending a full tablet report...
W20260808 02:13:17.156626 26802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:17.156917 26802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:17.157001 26802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:17.161271 26802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:17.161348 26802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:17.161435 26802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:17.166831 26802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab
--principal=oryx/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:41985
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=36959
--tserver_master_addrs=127.18.184.190:43577,127.18.184.189:44331,127.18.184.188:35335
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:17.167984 26802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:17.169055 26802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:17.179683 26809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:17.179683 26808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:17.179840 26811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:17.180248 26802 server_base.cc:1034] running on GCE node
I20260808 02:13:17.180750 26802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:17.181624 26802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:17.182803 26802 hybrid_clock.cc:648] HybridClock initialized: now 1786155197182789 us; error 26 us; skew 500 ppm
Aug 08 02:13:17 dist-test-slave-9gcw krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155197, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:17.186156 26802 init.cc:377] Logged in from keytab as oryx/127.18.184.130@KRBTEST.COM (short username oryx)
I20260808 02:13:17.187227 26802 webserver.cc:460] Webserver started at http://127.18.184.130:36959/ using document root <none> and password file <none>
I20260808 02:13:17.187877 26802 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:17.187990 26802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:17.192463 26802 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:17.195013 26819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:17.196446 26802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:13:17.196587 26802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "bc88b6ae2fd34d69b4d17083d79641e1"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-9gcw"
server_key: "650147b301d7e0c201d292c0b16f5b16"
server_key_iv: "54169c067fc16188bdaf668c296f2d85"
server_key_version: "encryptionkey@0"
I20260808 02:13:17.197273 26802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:17.202260 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.198717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35335 (local address 127.0.0.1:42184)
0808 02:13:17.198888 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.198904 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.199025 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.199344 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.199348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.199366 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:17.199475 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.199480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.200242 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.200248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.201004 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.201014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.201810 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.201814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.201902 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.201926 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.202019 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.202112 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 02:13:17.202260 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.198792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42184 (local address 127.18.184.188:35335)
0808 02:13:17.198928 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.198934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.198959 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:13:17.199143 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.199153 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.199212 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.199332 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:17.199586 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.200120 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.201185 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.201648 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.201801 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.201953 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.202035 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.202119 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:13:17.203140 26601 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 02:13:17.203593 26641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68f66e73d25e486080cfc178dc1df99f. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:17.203817 26641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:13:17.211261 26802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:17.215123 26802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:17.215368 26802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:17.216199 26802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:17.217316 26802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:17.217378 26802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:17.217450 26802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:17.217482 26802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:17.228025 26802 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:41985
I20260808 02:13:17.228052 26940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:41985 every 8 connection(s)
I20260808 02:13:17.229216 26802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:17.237016 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 26802
I20260808 02:13:17.237239 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 26093
Aug 08 02:13:17 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155197, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.188@KRBTEST.COM
Aug 08 02:13:17 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155197, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
Aug 08 02:13:17 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155197, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.189@KRBTEST.COM
I20260808 02:13:17.248406 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.231589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:33769 (local address 127.18.184.188:35335)
0808 02:13:17.231756 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.231761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.233125 (+ 1364us) server_negotiation.cc:407] Connection header received
0808 02:13:17.233644 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.233648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.233703 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.233767 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.235392 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.236279 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.237876 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.238413 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.238509 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.243238 (+ 4729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.243243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.243248 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.243289 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.245327 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.245993 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.245997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.245999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.246242 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.247325 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.247329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.247331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.247592 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.247686 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.248078 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.248218 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:13:17.248428 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.231149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:46985 (local address 127.18.184.190:43577)
0808 02:13:17.231311 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.231316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.232395 (+ 1079us) server_negotiation.cc:407] Connection header received
0808 02:13:17.233644 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.233648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.233703 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.233759 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.235335 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.235879 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.239460 (+ 3581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.239688 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.239812 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.242801 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.242805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.242808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.242840 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.245201 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.245993 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.245997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.245999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.246101 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.246716 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.246723 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.246729 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.247021 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.247132 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.247963 (+ 831us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.248134 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:13:17.249491 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:44559
--local_ip_for_outbound_sockets=127.18.184.131
--tserver_master_addrs=127.18.184.190:43577
--webserver_port=45121
--webserver_interface=127.18.184.131
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab
--principal=oryx/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.184.190:43577,127.18.184.189:44331
--tserver_master_addrs=127.18.184.190:43577,127.18.184.189:44331,127.18.184.188:35335 with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:13:17.249886 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.232914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:49785 (local address 127.18.184.189:44331)
0808 02:13:17.233036 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.233039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.233349 (+ 310us) server_negotiation.cc:407] Connection header received
0808 02:13:17.233626 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.233630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.233696 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.233805 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.235379 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.236277 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.242398 (+ 6121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.242865 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.243002 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.246229 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.246233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.246236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.246267 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.247958 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.248450 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.248454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.248456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.248545 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.248865 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.248868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.248869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.249093 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.249213 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.249575 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.249697 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:13:17.250497 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.232731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:44331 (local address 127.18.184.130:49785)
0808 02:13:17.233253 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.233265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.233448 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.234004 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.234011 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.234616 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:17.235195 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.235203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.236448 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.236451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.242253 (+ 5802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.242266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.243196 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.243204 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.243345 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.244034 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:17.244056 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:17.246001 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:17.248074 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.248076 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.248078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.248328 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.248665 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.248668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.248669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.248755 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.249285 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:17.249287 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:17.249372 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.250098 (+ 726us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.250253 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":154,"spinlock_wait_cycles":18432,"thread_start_us":107,"threads_started":1}
I20260808 02:13:17.250514 26947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.231601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35335 (local address 127.18.184.130:33769)
0808 02:13:17.233007 (+ 1406us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.233021 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.233394 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.234006 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.234013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.234612 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:17.235192 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.235203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.236563 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.236566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.237700 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.237716 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.238832 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.238838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.239096 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.239898 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:17.239927 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:17.243057 (+ 3130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:17.245456 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.245458 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.245460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.245868 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.247089 (+ 1221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.247093 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.247096 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.247196 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.247767 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:17.247770 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:17.247875 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.248665 (+ 790us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.249189 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1321,"mutex_wait_us":230,"spinlock_wait_cycles":95360,"thread_start_us":180,"threads_started":1}
I20260808 02:13:17.251281 26945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.231595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.130:46985)
0808 02:13:17.232194 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.232260 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.233394 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.234006 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.234013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.234607 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:17.235171 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.235185 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.236037 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.236040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.239324 (+ 3284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.239333 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.239814 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.239818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.239912 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.240441 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:17.240453 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:17.242608 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:17.245359 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.245368 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.245390 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.245868 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.246369 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.246374 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.246379 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.246575 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.247212 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:17.247223 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:17.247616 (+ 393us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.249090 (+ 1474us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.249311 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":31,"spinlock_wait_cycles":5376,"thread_start_us":652,"threads_started":1}
I20260808 02:13:17.251992 26941 heartbeater.cc:347] Connected to a master server at 127.18.184.188:35335
I20260808 02:13:17.252135 26943 heartbeater.cc:347] Connected to a master server at 127.18.184.189:44331
I20260808 02:13:17.252192 26942 heartbeater.cc:347] Connected to a master server at 127.18.184.190:43577
I20260808 02:13:17.252285 26941 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:17.252285 26942 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:17.252285 26943 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:17.252970 26943 heartbeater.cc:511] Master 127.18.184.189:44331 requested a full tablet report, sending...
I20260808 02:13:17.252970 26942 heartbeater.cc:511] Master 127.18.184.190:43577 requested a full tablet report, sending...
I20260808 02:13:17.252969 26941 heartbeater.cc:511] Master 127.18.184.188:35335 requested a full tablet report, sending...
I20260808 02:13:17.254266 25736 ts_manager.cc:194] Re-registered known tserver with Master: bc88b6ae2fd34d69b4d17083d79641e1 (127.18.184.130:41985)
I20260808 02:13:17.254367 26588 ts_manager.cc:194] Registered new tserver with Master: bc88b6ae2fd34d69b4d17083d79641e1 (127.18.184.130:41985)
I20260808 02:13:17.255126 25736 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:46985
I20260808 02:13:17.255789 26392 ts_manager.cc:194] Registered new tserver with Master: bc88b6ae2fd34d69b4d17083d79641e1 (127.18.184.130:41985)
W20260808 02:13:17.371975 26948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:17.372284 26948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:17.372359 26948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:17.376403 26948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:17.376495 26948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:17.376592 26948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:17.382216 26948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab
--principal=oryx/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:44559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=45121
--tserver_master_addrs=127.18.184.190:43577,127.18.184.189:44331,127.18.184.188:35335
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:17.383510 26948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:17.384608 26948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:17.395057 26958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:17.395138 26948 server_base.cc:1034] running on GCE node
W20260808 02:13:17.395078 26954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:17.395078 26955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:17.395907 26948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:17.396569 26948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:17.397782 26948 hybrid_clock.cc:648] HybridClock initialized: now 1786155197397761 us; error 32 us; skew 500 ppm
Aug 08 02:13:17 dist-test-slave-9gcw krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155197, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:17.400980 26948 init.cc:377] Logged in from keytab as oryx/127.18.184.131@KRBTEST.COM (short username oryx)
I20260808 02:13:17.402047 26948 webserver.cc:460] Webserver started at http://127.18.184.131:45121/ using document root <none> and password file <none>
I20260808 02:13:17.402693 26948 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:17.402802 26948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:17.407338 26948 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 02:13:17.409786 26967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:17.411175 26948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:17.411321 26948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "ce6ceb70486646e3a3e812c025e35998"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-9gcw"
server_key: "2c8c0495bdaa2afa149f00c84bfd7844"
server_key_iv: "c06dcd3646669045b9e45a0ba8d84d01"
server_key_version: "encryptionkey@0"
I20260808 02:13:17.411916 26948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:17.428503 26948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:17.432101 26948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:17.432323 26948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:17.432976 26948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:17.433938 26948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:17.434015 26948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:17.434088 26948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:17.434139 26948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:17.444000 26948 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:44559
I20260808 02:13:17.444036 27089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:44559 every 8 connection(s)
I20260808 02:13:17.445071 26948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:17.450169 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 26948
Aug 08 02:13:17 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155197, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.189@KRBTEST.COM
Aug 08 02:13:17 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155197, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.188@KRBTEST.COM
Aug 08 02:13:17 dist-test-slave-9gcw krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155197, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
I20260808 02:13:17.465363 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.446936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:49343 (local address 127.18.184.188:35335)
0808 02:13:17.447121 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.447125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.448457 (+ 1332us) server_negotiation.cc:407] Connection header received
0808 02:13:17.449518 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.449522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.449581 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.449662 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.451604 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.452440 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.454916 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.455256 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.455340 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.460440 (+ 5100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.460445 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.460448 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.460483 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.462316 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.463034 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.463041 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.463044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.463144 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.463635 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.463641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.463644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.463918 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.464018 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.464816 (+ 798us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.464961 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53,"spinlock_wait_cycles":3840}
I20260808 02:13:17.467077 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.453588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42192 (local address 127.18.184.188:35335)
0808 02:13:17.453875 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.453879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.454780 (+ 901us) server_negotiation.cc:407] Connection header received
0808 02:13:17.454823 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.454826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.454871 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.454945 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.457526 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.458340 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.459783 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.460186 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.460296 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.462247 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.462251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.462254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.462299 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.464443 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.464991 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.464994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.464996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.465093 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.465444 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.465447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.465449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.465869 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.465995 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.466653 (+ 658us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.466926 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"spinlock_wait_cycles":7936,"thread_start_us":105,"threads_started":1}
I20260808 02:13:17.467084 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.447436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.131:49141)
0808 02:13:17.448017 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.448084 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.449270 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.449871 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.449879 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.450547 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:17.451421 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.451443 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.452567 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.452570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.455035 (+ 2465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.455045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.455817 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.455824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.455949 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.456386 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:17.456406 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:17.459018 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:17.462877 (+ 3859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.462883 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.462889 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.463154 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.464210 (+ 1056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.464213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.464215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.464321 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.465101 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:17.465104 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:17.465249 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.465379 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.465806 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":1236,"spinlock_wait_cycles":1280,"thread_start_us":703,"threads_started":1}
I20260808 02:13:17.467383 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.452989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57704 (local address 127.18.184.189:44331)
0808 02:13:17.453447 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.453452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.455287 (+ 1835us) server_negotiation.cc:407] Connection header received
0808 02:13:17.455333 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.455335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.455403 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.457273 (+ 1870us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.457403 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.458301 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.459541 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.459927 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.460031 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.462085 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.462091 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.462094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.462126 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.464566 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.465111 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.465114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.465116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.465386 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.465815 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.465822 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.465824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.466080 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.466233 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.466956 (+ 723us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.467114 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":62,"threads_started":1}
I20260808 02:13:17.467785 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.447004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:49141 (local address 127.18.184.190:43577)
0808 02:13:17.447164 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.447170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.448234 (+ 1064us) server_negotiation.cc:407] Connection header received
0808 02:13:17.449486 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.449490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.449555 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.449690 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.451607 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.452380 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.455176 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.455542 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.455655 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.459197 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.459201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.459203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.459230 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.462689 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.463851 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.463857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.463861 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.463958 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.464521 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.464527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.464532 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.464765 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.464866 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.467498 (+ 2632us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.467625 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:13:17.468132 27095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.447436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35335 (local address 127.18.184.131:49343)
0808 02:13:17.448357 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.448368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.449281 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.449870 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.449880 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.450567 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:17.451424 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.451445 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.452596 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.452599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.454781 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.454789 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.455518 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.455522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.455628 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.456104 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:17.456117 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:17.460264 (+ 4147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:17.462458 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.462466 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.462486 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.462902 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.463287 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.463292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.463298 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.463507 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.464098 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:17.464106 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:17.464510 (+ 404us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.465849 (+ 1339us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.466030 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"mutex_wait_us":1038,"spinlock_wait_cycles":32512,"thread_start_us":134,"threads_started":1}
I20260808 02:13:17.469203 27091 heartbeater.cc:347] Connected to a master server at 127.18.184.190:43577
I20260808 02:13:17.469444 27091 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:17.470026 27091 heartbeater.cc:511] Master 127.18.184.190:43577 requested a full tablet report, sending...
I20260808 02:13:17.470350 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.448877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:43879 (local address 127.18.184.189:44331)
0808 02:13:17.449028 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.449033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.449049 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:13:17.449518 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.449522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.449592 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.449669 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.451604 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.452440 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.454137 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.454497 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.454593 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.459637 (+ 5044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.459641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.459644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.459671 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.464983 (+ 5312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.468745 (+ 3762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.468749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.468751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.468837 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.469181 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.469188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.469191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.469454 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.469559 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.470036 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.470171 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:13:17.470389 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.448493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:44331 (local address 127.18.184.131:43879)
0808 02:13:17.448734 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.448745 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.449281 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.449870 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.449880 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.450567 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:17.451427 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.451445 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.452853 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.452856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.454004 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.454013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.454775 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.454779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.454974 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.455893 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:17.455921 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:17.459486 (+ 3565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:17.468338 (+ 8852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.468341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.468345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.468618 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.468925 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.468928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.468929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.469022 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.469665 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:17.469669 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:17.469778 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.469961 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.470195 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":160,"spinlock_wait_cycles":14080,"thread_start_us":82,"threads_started":1}
I20260808 02:13:17.470587 27090 heartbeater.cc:347] Connected to a master server at 127.18.184.188:35335
I20260808 02:13:17.471143 27090 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:17.471297 25736 ts_manager.cc:194] Re-registered known tserver with Master: ce6ceb70486646e3a3e812c025e35998 (127.18.184.131:44559)
I20260808 02:13:17.471390 27090 heartbeater.cc:511] Master 127.18.184.188:35335 requested a full tablet report, sending...
I20260808 02:13:17.471721 27092 heartbeater.cc:347] Connected to a master server at 127.18.184.189:44331
I20260808 02:13:17.471804 27092 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:17.471961 27092 heartbeater.cc:511] Master 127.18.184.189:44331 requested a full tablet report, sending...
I20260808 02:13:17.472107 26588 ts_manager.cc:194] Registered new tserver with Master: ce6ceb70486646e3a3e812c025e35998 (127.18.184.131:44559)
I20260808 02:13:17.472163 25736 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:49141
I20260808 02:13:17.473234 26392 ts_manager.cc:194] Registered new tserver with Master: ce6ceb70486646e3a3e812c025e35998 (127.18.184.131:44559)
I20260808 02:13:17.476784 27103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.452619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55990 (local address 127.18.184.190:43577)
0808 02:13:17.453479 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.453497 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.454033 (+ 536us) server_negotiation.cc:407] Connection header received
0808 02:13:17.454184 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.454189 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.454246 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.454337 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.457713 (+ 3376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.458488 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.469602 (+ 11114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.469964 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.470053 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.473365 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.473369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.473371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.473397 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.475044 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.475546 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.475549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.475550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.475627 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.475948 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.475951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.475952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.476125 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.476251 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.476462 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.476568 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":91,"threads_started":1}
I20260808 02:13:17.479967 25736 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:17.483423 25736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:17.486323 26556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: NON_VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: true } } }
I20260808 02:13:17.486486 26556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:17.490079 26556 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f: attempting to promote NON_VOTER 6728af60b1d24132b3fe6e66c733de88 to VOTER
I20260808 02:13:17.490456 26556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } }
I20260808 02:13:17.491577 26601 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 LEARNER]: Refusing update from remote peer 68f66e73d25e486080cfc178dc1df99f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 02:13:17.491653 26402 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 FOLLOWER]: Refusing update from remote peer 68f66e73d25e486080cfc178dc1df99f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 02:13:17.492039 27112 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 02:13:17.492198 27113 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 02:13:17.494127 27112 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6728af60b1d24132b3fe6e66c733de88 (127.18.184.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } } }
I20260808 02:13:17.494215 26402 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6728af60b1d24132b3fe6e66c733de88 (127.18.184.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } } }
I20260808 02:13:17.494664 26601 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6728af60b1d24132b3fe6e66c733de88 (127.18.184.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } } }
I20260808 02:13:17.495954 27121 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } } }
I20260808 02:13:17.496161 27121 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:13:17.496455 26556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } } }
I20260808 02:13:17.496588 26556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:17.496731 26556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } } }
I20260808 02:13:17.496820 26556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:17.496778 27110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } } }
I20260808 02:13:17.496894 27110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:17.496850 26641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68f66e73d25e486080cfc178dc1df99f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } } }
I20260808 02:13:17.496968 26641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:13:17.505131 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.499477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:33407 (local address 127.0.0.1:41290)
0808 02:13:17.499644 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.499660 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.499744 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.500937 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.500940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.500961 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:17.501091 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.501098 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.502599 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.502603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.503780 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.503790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.504753 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.504757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.504839 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.504861 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.504926 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.504995 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 02:13:17.505216 27122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.499936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.18.184.129:33407)
0808 02:13:17.500268 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.500285 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.500316 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:13:17.500401 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.500407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.500652 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.501295 (+ 643us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:17.501436 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.502480 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.503952 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.504614 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.504759 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.504907 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.504961 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.505052 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":116,"threads_started":1}
I20260808 02:13:17.506233 27123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.500733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:41985 (local address 127.0.0.1:53360)
0808 02:13:17.501598 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.501611 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.501756 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.502361 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.502364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.502380 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:17.502461 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.502466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.503688 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.503692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.504727 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.504737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.505815 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.505819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.505912 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.505937 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.506016 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.506083 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":106,"threads_started":1}
I20260808 02:13:17.506711 27124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.500393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:44559 (local address 127.0.0.1:35274)
0808 02:13:17.500972 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.500987 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.501078 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.502060 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.502063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.502081 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:17.502173 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.502182 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.504148 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.504152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.505261 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.505271 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.506355 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.506358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.506447 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.506468 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.506518 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.506587 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":116,"threads_started":1}
I20260808 02:13:17.506923 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.500923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53360 (local address 127.18.184.130:41985)
0808 02:13:17.501381 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.501392 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.501717 (+ 325us) server_negotiation.cc:407] Connection header received
0808 02:13:17.501882 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.501887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.502080 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.502229 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:17.502709 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.503571 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.504864 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.505685 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.505807 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.505924 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.506649 (+ 725us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.506774 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":104,"threads_started":1}
I20260808 02:13:17.507334 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.501199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35274 (local address 127.18.184.131:44559)
0808 02:13:17.501525 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.501538 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.501561 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:13:17.501644 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.501652 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.501871 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.502084 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:17.502316 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.504022 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.505397 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.506183 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.506889 (+ 706us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.507091 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.507145 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.507220 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":125,"threads_started":1}
I20260808 02:13:17.508464 26721 tablet_service.cc:1467] Processing CreateTablet for tablet 43cd9bc40f6f4edf9e3d91e0a8063ff4 (DEFAULT_TABLE table=test-table [id=8375a296727442dba8ab7aecccc0cba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:17.509697 26721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43cd9bc40f6f4edf9e3d91e0a8063ff4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:17.512079 26873 tablet_service.cc:1467] Processing CreateTablet for tablet 43cd9bc40f6f4edf9e3d91e0a8063ff4 (DEFAULT_TABLE table=test-table [id=8375a296727442dba8ab7aecccc0cba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:17.512079 27018 tablet_service.cc:1467] Processing CreateTablet for tablet 43cd9bc40f6f4edf9e3d91e0a8063ff4 (DEFAULT_TABLE table=test-table [id=8375a296727442dba8ab7aecccc0cba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:17.513186 26873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43cd9bc40f6f4edf9e3d91e0a8063ff4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:17.513196 27018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43cd9bc40f6f4edf9e3d91e0a8063ff4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:17.521194 27128 tablet_bootstrap.cc:492] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1: Bootstrap starting.
I20260808 02:13:17.521900 27127 tablet_bootstrap.cc:492] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575: Bootstrap starting.
I20260808 02:13:17.523423 27128 tablet_bootstrap.cc:654] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:17.523612 27127 tablet_bootstrap.cc:654] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:17.524207 27128 log.cc:824] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:17.524286 27127 log.cc:824] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:17.524988 27129 tablet_bootstrap.cc:492] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998: Bootstrap starting.
I20260808 02:13:17.526235 27127 tablet_bootstrap.cc:492] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575: No bootstrap required, opened a new log
I20260808 02:13:17.526414 27128 tablet_bootstrap.cc:492] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1: No bootstrap required, opened a new log
I20260808 02:13:17.526448 27127 ts_tablet_manager.cc:1397] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 02:13:17.526597 27128 ts_tablet_manager.cc:1397] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 02:13:17.527282 27129 tablet_bootstrap.cc:654] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:17.528153 27129 log.cc:824] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:17.530948 27129 tablet_bootstrap.cc:492] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998: No bootstrap required, opened a new log
I20260808 02:13:17.531126 27129 ts_tablet_manager.cc:1397] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:13:17.531080 27127 raft_consensus.cc:348] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } }
I20260808 02:13:17.531337 27127 raft_consensus.cc:374] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:17.531390 27127 raft_consensus.cc:729] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236b94a7ada74cc28c42bf86c2db8575, State: Initialized, Role: FOLLOWER
I20260808 02:13:17.531304 27128 raft_consensus.cc:348] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } }
I20260808 02:13:17.531543 27128 raft_consensus.cc:374] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:17.531594 27128 raft_consensus.cc:729] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc88b6ae2fd34d69b4d17083d79641e1, State: Initialized, Role: FOLLOWER
I20260808 02:13:17.531883 27127 consensus_queue.cc:260] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } }
I20260808 02:13:17.532047 27128 consensus_queue.cc:260] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } }
I20260808 02:13:17.532837 27127 ts_tablet_manager.cc:1428] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260808 02:13:17.532982 27128 ts_tablet_manager.cc:1428] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260808 02:13:17.534507 26794 heartbeater.cc:503] Master 127.18.184.190:43577 was elected leader, sending a full tablet report...
I20260808 02:13:17.535765 26942 heartbeater.cc:503] Master 127.18.184.190:43577 was elected leader, sending a full tablet report...
I20260808 02:13:17.538911 27133 raft_consensus.cc:482] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:17.539072 27133 raft_consensus.cc:504] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } }
I20260808 02:13:17.540452 27133 leader_election.cc:288] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce6ceb70486646e3a3e812c025e35998 (127.18.184.131:44559), bc88b6ae2fd34d69b4d17083d79641e1 (127.18.184.130:41985)
I20260808 02:13:17.544878 27129 raft_consensus.cc:348] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } }
I20260808 02:13:17.545230 27129 raft_consensus.cc:374] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:17.545295 27129 raft_consensus.cc:729] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce6ceb70486646e3a3e812c025e35998, State: Initialized, Role: FOLLOWER
I20260808 02:13:17.545621 26801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.540631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:44559 (local address 127.18.184.129:55639)
0808 02:13:17.540775 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.540791 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.540872 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.541938 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.541941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.541957 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:17.542063 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.542069 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.543208 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.543211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.544330 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.544340 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.545173 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.545177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.545278 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.545311 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.545388 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.545462 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 02:13:17.545831 26797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.540913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:41985 (local address 127.18.184.129:43539)
0808 02:13:17.541236 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.541251 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.541324 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.542059 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.542062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.542081 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:17.542204 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.542214 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.543669 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.543672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.544639 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.544650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.545418 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.545422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.545539 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.545564 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.545644 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.545713 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254}
I20260808 02:13:17.545938 27129 consensus_queue.cc:260] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } }
I20260808 02:13:17.546154 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.541628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:43539 (local address 127.18.184.130:41985)
0808 02:13:17.541751 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.541756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.541769 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:13:17.541809 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.541812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.541852 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.541923 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:17.542342 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.543105 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.544761 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.545281 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.545408 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.545452 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.545967 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.546046 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 02:13:17.546409 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.541478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:55639 (local address 127.18.184.131:44559)
0808 02:13:17.541604 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.541609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.541623 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:13:17.541671 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.541674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.541719 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.541787 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:17.542188 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.543006 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.544459 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.545019 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.545168 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.545240 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.546152 (+ 912us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.546266 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 02:13:17.546890 27040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cd9bc40f6f4edf9e3d91e0a8063ff4" candidate_uuid: "236b94a7ada74cc28c42bf86c2db8575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce6ceb70486646e3a3e812c025e35998" is_pre_election: true
I20260808 02:13:17.546960 26895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cd9bc40f6f4edf9e3d91e0a8063ff4" candidate_uuid: "236b94a7ada74cc28c42bf86c2db8575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" is_pre_election: true
I20260808 02:13:17.547194 27040 raft_consensus.cc:2454] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 236b94a7ada74cc28c42bf86c2db8575 in term 0.
I20260808 02:13:17.547241 26895 raft_consensus.cc:2454] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 236b94a7ada74cc28c42bf86c2db8575 in term 0.
I20260808 02:13:17.547588 27129 ts_tablet_manager.cc:1428] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998: Time spent starting tablet: real 0.016s user 0.008s sys 0.000s
I20260808 02:13:17.547660 26671 leader_election.cc:302] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 236b94a7ada74cc28c42bf86c2db8575, ce6ceb70486646e3a3e812c025e35998; no voters:
I20260808 02:13:17.547972 27091 heartbeater.cc:503] Master 127.18.184.190:43577 was elected leader, sending a full tablet report...
I20260808 02:13:17.548213 27133 raft_consensus.cc:2781] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:17.548290 27133 raft_consensus.cc:482] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:17.548332 27133 raft_consensus.cc:3037] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:17.551847 27133 raft_consensus.cc:504] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } }
I20260808 02:13:17.552248 27133 leader_election.cc:288] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [CANDIDATE]: Term 1 election: Requested vote from peers ce6ceb70486646e3a3e812c025e35998 (127.18.184.131:44559), bc88b6ae2fd34d69b4d17083d79641e1 (127.18.184.130:41985)
I20260808 02:13:17.552594 27040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cd9bc40f6f4edf9e3d91e0a8063ff4" candidate_uuid: "236b94a7ada74cc28c42bf86c2db8575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce6ceb70486646e3a3e812c025e35998"
I20260808 02:13:17.552697 27040 raft_consensus.cc:3037] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:17.552654 26895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cd9bc40f6f4edf9e3d91e0a8063ff4" candidate_uuid: "236b94a7ada74cc28c42bf86c2db8575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc88b6ae2fd34d69b4d17083d79641e1"
I20260808 02:13:17.552802 26895 raft_consensus.cc:3037] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:17.554193 27040 raft_consensus.cc:2454] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 236b94a7ada74cc28c42bf86c2db8575 in term 1.
I20260808 02:13:17.554338 26895 raft_consensus.cc:2454] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 236b94a7ada74cc28c42bf86c2db8575 in term 1.
I20260808 02:13:17.554492 26671 leader_election.cc:302] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 236b94a7ada74cc28c42bf86c2db8575, ce6ceb70486646e3a3e812c025e35998; no voters:
I20260808 02:13:17.554715 27133 raft_consensus.cc:2781] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:17.554826 27133 raft_consensus.cc:686] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 1 LEADER]: Becoming Leader. State: Replica: 236b94a7ada74cc28c42bf86c2db8575, State: Running, Role: LEADER
I20260808 02:13:17.555161 27133 consensus_queue.cc:237] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } }
I20260808 02:13:17.559118 25738 catalog_manager.cc:5495] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 reported cstate change: term changed from 0 to 1, leader changed from <none> to 236b94a7ada74cc28c42bf86c2db8575 (127.18.184.129). New cstate: current_term: 1 leader_uuid: "236b94a7ada74cc28c42bf86c2db8575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:17.588282 27122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.584554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41302 (local address 127.18.184.129:33407)
0808 02:13:17.584817 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.584821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.584843 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:13:17.584931 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.584934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.584993 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.585096 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:17.585405 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.586003 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.586845 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.587055 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.587132 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.587413 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:17.587848 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:17.587940 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.588093 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.588158 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260808 02:13:17.594467 26895 raft_consensus.cc:1261] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1 [term 1 FOLLOWER]: Refusing update from remote peer 236b94a7ada74cc28c42bf86c2db8575: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:17.594477 27040 raft_consensus.cc:1261] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998 [term 1 FOLLOWER]: Refusing update from remote peer 236b94a7ada74cc28c42bf86c2db8575: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:17.595261 27133 consensus_queue.cc:1035] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:17.595496 27135 consensus_queue.cc:1035] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:17.615015 25738 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:17.615216 25738 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:17.620932 25738 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68f66e73d25e486080cfc178dc1df99f: Sending DeleteTablet for 3 replicas of tablet 43cd9bc40f6f4edf9e3d91e0a8063ff4
I20260808 02:13:17.621975 26721 tablet_service.cc:1514] Processing DeleteTablet for tablet 43cd9bc40f6f4edf9e3d91e0a8063ff4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:17 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:41290
I20260808 02:13:17.622000 27018 tablet_service.cc:1514] Processing DeleteTablet for tablet 43cd9bc40f6f4edf9e3d91e0a8063ff4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:17 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:35274
I20260808 02:13:17.623340 26873 tablet_service.cc:1514] Processing DeleteTablet for tablet 43cd9bc40f6f4edf9e3d91e0a8063ff4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:17 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:53360
I20260808 02:13:17.623979 27154 tablet_replica.cc:330] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1: stopping tablet replica
I20260808 02:13:17.624399 27154 raft_consensus.cc:2229] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:17.624784 27154 raft_consensus.cc:2258] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:17.625013 27153 tablet_replica.cc:330] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998: stopping tablet replica
I20260808 02:13:17.625433 27153 raft_consensus.cc:2229] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:17.625803 27153 raft_consensus.cc:2258] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:17.626264 27152 tablet_replica.cc:330] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575: stopping tablet replica
I20260808 02:13:17.626593 27152 raft_consensus.cc:2229] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:17.627013 27152 raft_consensus.cc:2258] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:17.627543 27153 ts_tablet_manager.cc:1905] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:17.627727 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 25705
I20260808 02:13:17.631831 27153 ts_tablet_manager.cc:1918] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:17.632737 27154 ts_tablet_manager.cc:1905] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:17.635560 27154 ts_tablet_manager.cc:1918] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:17.635783 27153 log.cc:1196] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/43cd9bc40f6f4edf9e3d91e0a8063ff4
I20260808 02:13:17.636214 27153 ts_tablet_manager.cc:1939] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P ce6ceb70486646e3a3e812c025e35998: Deleting consensus metadata
I20260808 02:13:17.637811 27154 log.cc:1196] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/43cd9bc40f6f4edf9e3d91e0a8063ff4
I20260808 02:13:17.638283 27154 ts_tablet_manager.cc:1939] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P bc88b6ae2fd34d69b4d17083d79641e1: Deleting consensus metadata
I20260808 02:13:17.643600 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.625633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57710 (local address 127.18.184.189:44331)
0808 02:13:17.625906 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.625912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.627702 (+ 1790us) server_negotiation.cc:407] Connection header received
0808 02:13:17.627935 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.627942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.628012 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.628074 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.631603 (+ 3529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.632883 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.634803 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.635396 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.635522 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.638972 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.638980 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.638988 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.639036 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.641183 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.641708 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.641712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.641714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.641803 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.642166 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.642173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.642178 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.642525 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.642695 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.643161 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.643328 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260808 02:13:17.651151 27152 ts_tablet_manager.cc:1905] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 02:13:17.651525 26820 connection.cc:535] server connection from 127.0.0.1:53360 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 02:13:17.652064 26971 connection.cc:535] server connection from 127.0.0.1:35274 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 02:13:17.654498 27152 ts_tablet_manager.cc:1918] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:17.654716 27152 log.cc:1196] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-0/wal/wals/43cd9bc40f6f4edf9e3d91e0a8063ff4
I20260808 02:13:17.655153 27152 ts_tablet_manager.cc:1939] T 43cd9bc40f6f4edf9e3d91e0a8063ff4 P 236b94a7ada74cc28c42bf86c2db8575: Deleting consensus metadata
I20260808 02:13:17.656391 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.625757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42194 (local address 127.18.184.188:35335)
0808 02:13:17.626015 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.626020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.645461 (+ 19441us) server_negotiation.cc:407] Connection header received
0808 02:13:17.645544 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.645550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.645623 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.645750 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.646535 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.647797 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.648976 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.649499 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.649639 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.651843 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.651848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.651851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.651883 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.654014 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.654621 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.654629 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.654634 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.654757 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.655151 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.655155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.655181 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.655469 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.655605 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.656066 (+ 461us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.656218 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176}
W20260808 02:13:17.656431 26673 connection.cc:441] server connection from 127.0.0.1:41290 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41290 (request call id 1) could send its response
I20260808 02:13:17.668449 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.658383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57714 (local address 127.18.184.189:44331)
0808 02:13:17.658521 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.658528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.658696 (+ 168us) server_negotiation.cc:407] Connection header received
0808 02:13:17.658826 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.658829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.658890 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.658973 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.659824 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.660630 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.661885 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.662240 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.662350 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.664299 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.664304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.664307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.664338 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.666340 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.666957 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.666962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.666964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.667069 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.667396 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.667403 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.667408 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.667653 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.667815 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.668094 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.668229 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:13:17.668475 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.658025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42196 (local address 127.18.184.188:35335)
0808 02:13:17.658181 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.658186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.658199 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:13:17.658233 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.658236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.658608 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.658714 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.659800 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.660371 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.661650 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.662007 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.662073 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.664314 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.664318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.664321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.664357 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.666498 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.667044 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.667048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.667050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.667147 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.667466 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.667470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.667472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.667704 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.667829 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.668176 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.668307 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 02:13:18.609828 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:18.609105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.131:36837)
0808 02:13:18.609454 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:18.609581 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":100,"threads_started":1}
W20260808 02:13:18.610176 27091 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:43577 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
I20260808 02:13:18.611797 27164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:18.611260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.129:59309)
0808 02:13:18.611557 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:18.611672 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":95,"threads_started":1}
W20260808 02:13:18.612095 26794 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:43577 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
I20260808 02:13:18.627727 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:18.627231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.130:51657)
0808 02:13:18.627515 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:18.627606 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":84,"threads_started":1}
W20260808 02:13:18.628187 26942 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:43577 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
I20260808 02:13:19.121395 27166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 68f66e73d25e486080cfc178dc1df99f)
I20260808 02:13:19.121667 27166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } }
I20260808 02:13:19.122716 27166 leader_election.cc:288] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 68f66e73d25e486080cfc178dc1df99f (127.18.184.190:43577), f9a3d956916147c9ad89c32fc1fe6317 (127.18.184.189:44331)
I20260808 02:13:19.122861 27167 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 68f66e73d25e486080cfc178dc1df99f)
I20260808 02:13:19.123075 27167 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } }
I20260808 02:13:19.123481 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.122994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:56470)
0808 02:13:19.123255 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.123364 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":70,"threads_started":1}
W20260808 02:13:19.123672 26582 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
I20260808 02:13:19.124159 27167 leader_election.cc:288] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 68f66e73d25e486080cfc178dc1df99f (127.18.184.190:43577), 6728af60b1d24132b3fe6e66c733de88 (127.18.184.188:35335)
I20260808 02:13:19.124542 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.124295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:56490)
0808 02:13:19.124402 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.124463 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 02:13:19.124699 26582 leader_election.cc:334] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 68f66e73d25e486080cfc178dc1df99f (127.18.184.190:43577): Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
I20260808 02:13:19.124948 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.124267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:56486)
0808 02:13:19.124711 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.124841 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":60,"threads_started":1}
W20260808 02:13:19.125216 26375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
I20260808 02:13:19.126180 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.125947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:56500)
0808 02:13:19.126068 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.126121 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 02:13:19.126288 26375 leader_election.cc:334] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 68f66e73d25e486080cfc178dc1df99f (127.18.184.190:43577): Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
I20260808 02:13:19.128070 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.123457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57722 (local address 127.18.184.189:44331)
0808 02:13:19.123850 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:19.123855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:19.124154 (+ 299us) server_negotiation.cc:407] Connection header received
0808 02:13:19.124383 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:19.124386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:19.124440 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:19.124500 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:19.124961 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.125793 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.127094 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.127514 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.127655 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.127709 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:19.127886 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 02:13:19.127956 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":75,"threads_started":1}
I20260808 02:13:19.128100 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.123344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:44331 (local address 127.0.0.1:57722)
0808 02:13:19.124066 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.124082 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:19.124242 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:19.124671 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:19.124674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:19.124693 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:19.124821 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.124829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.125921 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.125924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.126964 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.126974 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.127650 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.127654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.127753 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.127776 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:13:19.127868 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 02:13:19.127947 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":52,"threads_started":1}
I20260808 02:13:19.128544 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6728af60b1d24132b3fe6e66c733de88" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f9a3d956916147c9ad89c32fc1fe6317" is_pre_election: true
I20260808 02:13:19.128670 27173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.124616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35335 (local address 127.0.0.1:42198)
0808 02:13:19.124966 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.124979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:19.125054 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:19.125533 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:19.125536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:19.125559 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:19.125661 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.125668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.126652 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.126657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.127486 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.127495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.128257 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.128262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.128356 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.128380 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:13:19.128428 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 02:13:19.128501 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":120,"threads_started":1}
I20260808 02:13:19.128788 26402 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6728af60b1d24132b3fe6e66c733de88 in term 1.
I20260808 02:13:19.128966 27174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.125037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42198 (local address 127.18.184.188:35335)
0808 02:13:19.125309 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:19.125317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:19.125330 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:13:19.125367 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:19.125370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:19.125408 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:19.125490 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:19.125799 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.126541 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.127615 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.128084 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.128645 (+ 561us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.128708 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:19.128749 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 02:13:19.128828 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":128,"threads_started":1}
I20260808 02:13:19.129212 26582 leader_election.cc:302] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6728af60b1d24132b3fe6e66c733de88, f9a3d956916147c9ad89c32fc1fe6317; no voters: 68f66e73d25e486080cfc178dc1df99f
I20260808 02:13:19.129371 26605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f9a3d956916147c9ad89c32fc1fe6317" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6728af60b1d24132b3fe6e66c733de88" is_pre_election: true
I20260808 02:13:19.129485 27166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 02:13:19.129534 27166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 68f66e73d25e486080cfc178dc1df99f)
I20260808 02:13:19.129577 27166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:13:19.130618 27166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } }
I20260808 02:13:19.130877 26605 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f9a3d956916147c9ad89c32fc1fe6317 in current term 2: Already voted for candidate 6728af60b1d24132b3fe6e66c733de88 in this term.
I20260808 02:13:19.131037 27166 leader_election.cc:288] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [CANDIDATE]: Term 2 election: Requested vote from peers 68f66e73d25e486080cfc178dc1df99f (127.18.184.190:43577), f9a3d956916147c9ad89c32fc1fe6317 (127.18.184.189:44331)
I20260808 02:13:19.131451 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.131181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:56516)
0808 02:13:19.131315 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.131380 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 02:13:19.131613 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6728af60b1d24132b3fe6e66c733de88" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f9a3d956916147c9ad89c32fc1fe6317"
I20260808 02:13:19.131876 26402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:13:19.131888 26373 leader_election.cc:302] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9a3d956916147c9ad89c32fc1fe6317; no voters: 6728af60b1d24132b3fe6e66c733de88, 68f66e73d25e486080cfc178dc1df99f
I20260808 02:13:19.132198 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.131891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:56524)
0808 02:13:19.132079 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.132135 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 02:13:19.132355 26582 leader_election.cc:334] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 68f66e73d25e486080cfc178dc1df99f (127.18.184.190:43577): Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
I20260808 02:13:19.133108 26402 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6728af60b1d24132b3fe6e66c733de88 in term 2.
I20260808 02:13:19.133276 27167 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:13:19.133486 26582 leader_election.cc:302] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6728af60b1d24132b3fe6e66c733de88, f9a3d956916147c9ad89c32fc1fe6317; no voters: 68f66e73d25e486080cfc178dc1df99f
I20260808 02:13:19.133922 27166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:13:19.134172 27166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [term 2 LEADER]: Becoming Leader. State: Replica: 6728af60b1d24132b3fe6e66c733de88, State: Running, Role: LEADER
I20260808 02:13:19.134467 27166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } }
I20260808 02:13:19.135879 27176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6728af60b1d24132b3fe6e66c733de88. Latest consensus state: current_term: 2 leader_uuid: "6728af60b1d24132b3fe6e66c733de88" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } } }
I20260808 02:13:19.136011 27176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:19.136346 27178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:19.139437 27178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:19.139930 27178 catalog_manager.cc:1259] Loaded cluster ID: 43c29d2eb3a54bbc9d7df32a70270782
I20260808 02:13:19.139981 27178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:19.140584 27178 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:19.141021 27178 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: Loaded TSK: 0
I20260808 02:13:19.141809 27178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:19.203382 26588 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:19.206677 26588 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:19.209956 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.209695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:56540)
0808 02:13:19.209833 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.209885 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 02:13:19.209955 26402 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [term 2 FOLLOWER]: Refusing update from remote peer 6728af60b1d24132b3fe6e66c733de88: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 02:13:19.210549 27176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 02:13:19.210901 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.210608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.0.0.1:56548)
0808 02:13:19.210731 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.210840 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 02:13:19.211109 26582 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 -> Peer 68f66e73d25e486080cfc178dc1df99f (127.18.184.190:43577): Couldn't send request to peer 68f66e73d25e486080cfc178dc1df99f. Status: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 02:13:19.212137 27167 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6728af60b1d24132b3fe6e66c733de88. Latest consensus state: current_term: 2 leader_uuid: "6728af60b1d24132b3fe6e66c733de88" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } } }
I20260808 02:13:19.212301 27167 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:13:19.213408 27166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6728af60b1d24132b3fe6e66c733de88" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } } }
I20260808 02:13:19.213534 27166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:19.213846 27167 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6728af60b1d24132b3fe6e66c733de88" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "68f66e73d25e486080cfc178dc1df99f" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43577 } } peers { permanent_uuid: "f9a3d956916147c9ad89c32fc1fe6317" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 44331 } attrs { promote: false } } peers { permanent_uuid: "6728af60b1d24132b3fe6e66c733de88" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35335 } attrs { promote: false } } }
I20260808 02:13:19.213961 27167 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a3d956916147c9ad89c32fc1fe6317 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:13:19.223968 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.219594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:33407 (local address 127.0.0.1:41304)
0808 02:13:19.219839 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.219856 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:19.219949 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:19.220480 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:19.220484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:19.220499 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:19.220665 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.220672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.222002 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.222006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.222845 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.222854 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.223557 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.223561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.223671 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.223701 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 02:13:19.223749 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 02:13:19.223825 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":77}
I20260808 02:13:19.224599 27185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.219756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.18.184.129:33407)
0808 02:13:19.220206 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:19.220214 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:19.220230 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:13:19.220290 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:19.220293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:19.220343 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:19.220992 (+ 649us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:19.221102 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.221864 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.222976 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.223404 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.224277 (+ 873us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.224335 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:19.224372 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 02:13:19.224455 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":72,"threads_started":1}
I20260808 02:13:19.224936 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.220005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:41985 (local address 127.0.0.1:53362)
0808 02:13:19.220095 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.220111 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:19.220219 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:19.220665 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:19.220667 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:19.220685 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:19.220777 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.220783 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.221972 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.221976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.223367 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.223378 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.224447 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.224451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.224547 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.224574 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:13:19.224658 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 02:13:19.224733 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":24}
I20260808 02:13:19.225174 26721 tablet_service.cc:1467] Processing CreateTablet for tablet b598cc581fc64de0a5ac008b565efb00 (DEFAULT_TABLE table=test-table [id=dd4e70d1cc5a42828b69bf55e51fd6f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:19.225477 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.220111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.18.184.130:41985)
0808 02:13:19.220375 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:19.220382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:19.220398 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:13:19.220455 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:19.220460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:19.220545 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:19.220886 (+ 341us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:19.221020 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.221854 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.223509 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.224324 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.224440 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.224484 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:19.224865 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 02:13:19.224975 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":86,"threads_started":1}
I20260808 02:13:19.225500 26721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b598cc581fc64de0a5ac008b565efb00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:19.225809 27188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.220228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:44559 (local address 127.0.0.1:35278)
0808 02:13:19.220508 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.220523 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:19.220900 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:19.221619 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:19.221622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:19.221639 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:19.221715 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.221719 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.222921 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.222925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.224121 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.224131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.225334 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.225341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.225443 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.225474 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:13:19.225573 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 02:13:19.225673 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":122,"threads_started":1}
I20260808 02:13:19.226181 27189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.220688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35278 (local address 127.18.184.131:44559)
0808 02:13:19.220948 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:19.220956 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:19.220972 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:13:19.221208 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:19.221212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:19.221273 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:19.221481 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:19.221843 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.222774 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.224273 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.225019 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.225180 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.225237 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:19.225749 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 02:13:19.225842 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":87,"threads_started":1}
I20260808 02:13:19.226763 27018 tablet_service.cc:1467] Processing CreateTablet for tablet b598cc581fc64de0a5ac008b565efb00 (DEFAULT_TABLE table=test-table [id=dd4e70d1cc5a42828b69bf55e51fd6f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:19.227061 27018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b598cc581fc64de0a5ac008b565efb00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:19.228336 27190 tablet_bootstrap.cc:492] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575: Bootstrap starting.
I20260808 02:13:19.228585 26873 tablet_service.cc:1467] Processing CreateTablet for tablet b598cc581fc64de0a5ac008b565efb00 (DEFAULT_TABLE table=test-table [id=dd4e70d1cc5a42828b69bf55e51fd6f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:19.228860 26873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b598cc581fc64de0a5ac008b565efb00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:19.229487 27190 tablet_bootstrap.cc:654] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:19.230968 27190 tablet_bootstrap.cc:492] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575: No bootstrap required, opened a new log
I20260808 02:13:19.231048 27190 ts_tablet_manager.cc:1397] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:19.231439 27193 tablet_bootstrap.cc:492] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1: Bootstrap starting.
I20260808 02:13:19.231482 27190 raft_consensus.cc:348] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } }
I20260808 02:13:19.231556 27191 tablet_bootstrap.cc:492] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998: Bootstrap starting.
I20260808 02:13:19.231591 27190 raft_consensus.cc:374] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:19.231632 27190 raft_consensus.cc:729] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236b94a7ada74cc28c42bf86c2db8575, State: Initialized, Role: FOLLOWER
I20260808 02:13:19.231755 27190 consensus_queue.cc:260] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } }
I20260808 02:13:19.232072 27190 ts_tablet_manager.cc:1428] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:19.232388 27193 tablet_bootstrap.cc:654] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:19.232489 27191 tablet_bootstrap.cc:654] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:19.233234 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.232732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.129:38847)
0808 02:13:19.233002 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.233094 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":120,"threads_started":1}
I20260808 02:13:19.233731 27193 tablet_bootstrap.cc:492] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1: No bootstrap required, opened a new log
I20260808 02:13:19.233736 27191 tablet_bootstrap.cc:492] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998: No bootstrap required, opened a new log
I20260808 02:13:19.233811 27193 ts_tablet_manager.cc:1397] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:19.233811 27191 ts_tablet_manager.cc:1397] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:19.234233 27193 raft_consensus.cc:348] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } }
I20260808 02:13:19.234233 27191 raft_consensus.cc:348] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } }
I20260808 02:13:19.234338 27191 raft_consensus.cc:374] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:19.234338 27193 raft_consensus.cc:374] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:19.234372 27191 raft_consensus.cc:729] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce6ceb70486646e3a3e812c025e35998, State: Initialized, Role: FOLLOWER
I20260808 02:13:19.234372 27193 raft_consensus.cc:729] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc88b6ae2fd34d69b4d17083d79641e1, State: Initialized, Role: FOLLOWER
I20260808 02:13:19.234503 27193 consensus_queue.cc:260] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } }
I20260808 02:13:19.234503 27191 consensus_queue.cc:260] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } }
I20260808 02:13:19.234830 27191 ts_tablet_manager.cc:1428] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:19.234830 27193 ts_tablet_manager.cc:1428] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:19.236366 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.235858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.130:52997)
0808 02:13:19.236186 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.236250 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":114,"threads_started":1}
I20260808 02:13:19.237836 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.237411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.131:37561)
0808 02:13:19.237646 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.237711 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
I20260808 02:13:19.241179 26793 heartbeater.cc:503] Master 127.18.184.188:35335 was elected leader, sending a full tablet report...
I20260808 02:13:19.241739 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.241433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.129:42003)
0808 02:13:19.241580 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.241642 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 02:13:19.248350 27198 raft_consensus.cc:482] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:19.248548 27198 raft_consensus.cc:504] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } }
I20260808 02:13:19.249915 27198 leader_election.cc:288] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 236b94a7ada74cc28c42bf86c2db8575 (127.18.184.129:33407), ce6ceb70486646e3a3e812c025e35998 (127.18.184.131:44559)
I20260808 02:13:19.253655 27185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.250194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:60877 (local address 127.18.184.129:33407)
0808 02:13:19.250327 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:19.250334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:19.250354 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:13:19.250616 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:19.250620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:19.250669 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:19.250778 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:19.251050 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.251589 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.252658 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.253072 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.253234 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.253292 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:19.253467 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 02:13:19.253537 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:13:19.253656 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.250090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:33407 (local address 127.18.184.130:60877)
0808 02:13:19.250216 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.250233 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:19.250345 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:19.250791 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:19.250794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:19.250813 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:19.250936 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.250941 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.251705 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.251709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.252530 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.252539 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.253214 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.253218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.253330 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.253360 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 02:13:19.253451 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 02:13:19.253519 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"spinlock_wait_cycles":8576}
I20260808 02:13:19.254190 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.250367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:44559 (local address 127.18.184.130:49073)
0808 02:13:19.250697 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.250720 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:19.250848 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:19.251121 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:19.251124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:19.251141 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:19.251255 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.251266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.252065 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.252068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.252905 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:19.252914 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.253725 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.253729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:19.253826 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.253849 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:13:19.253955 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 02:13:19.254024 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":170,"threads_started":1}
I20260808 02:13:19.254240 27189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.250484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:49073 (local address 127.18.184.131:44559)
0808 02:13:19.250614 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:19.250619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:19.250811 (+ 192us) server_negotiation.cc:407] Connection header received
0808 02:13:19.250948 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:19.250951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:19.251007 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:19.251109 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:19.251378 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.251934 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.253049 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.253564 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.253730 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.253784 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:19.253999 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 02:13:19.254100 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:13:19.254478 26746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b598cc581fc64de0a5ac008b565efb00" candidate_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236b94a7ada74cc28c42bf86c2db8575" is_pre_election: true
I20260808 02:13:19.254807 26746 raft_consensus.cc:2454] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc88b6ae2fd34d69b4d17083d79641e1 in term 0.
I20260808 02:13:19.254861 27040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b598cc581fc64de0a5ac008b565efb00" candidate_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce6ceb70486646e3a3e812c025e35998" is_pre_election: true
I20260808 02:13:19.255050 27040 raft_consensus.cc:2454] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc88b6ae2fd34d69b4d17083d79641e1 in term 0.
I20260808 02:13:19.255333 26822 leader_election.cc:302] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 236b94a7ada74cc28c42bf86c2db8575, bc88b6ae2fd34d69b4d17083d79641e1; no voters:
I20260808 02:13:19.255671 27198 raft_consensus.cc:2781] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:19.255738 27198 raft_consensus.cc:482] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:19.255782 27198 raft_consensus.cc:3037] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:19.257007 27198 raft_consensus.cc:504] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } }
I20260808 02:13:19.257428 27198 leader_election.cc:288] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [CANDIDATE]: Term 1 election: Requested vote from peers 236b94a7ada74cc28c42bf86c2db8575 (127.18.184.129:33407), ce6ceb70486646e3a3e812c025e35998 (127.18.184.131:44559)
I20260808 02:13:19.257774 26746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b598cc581fc64de0a5ac008b565efb00" candidate_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236b94a7ada74cc28c42bf86c2db8575"
I20260808 02:13:19.257846 27040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b598cc581fc64de0a5ac008b565efb00" candidate_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce6ceb70486646e3a3e812c025e35998"
I20260808 02:13:19.257965 26746 raft_consensus.cc:3037] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:19.257973 27040 raft_consensus.cc:3037] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:19.258987 27040 raft_consensus.cc:2454] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc88b6ae2fd34d69b4d17083d79641e1 in term 1.
I20260808 02:13:19.259119 26746 raft_consensus.cc:2454] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc88b6ae2fd34d69b4d17083d79641e1 in term 1.
I20260808 02:13:19.259326 26822 leader_election.cc:302] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc88b6ae2fd34d69b4d17083d79641e1, ce6ceb70486646e3a3e812c025e35998; no voters:
I20260808 02:13:19.259516 27198 raft_consensus.cc:2781] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:19.259755 27198 raft_consensus.cc:686] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 1 LEADER]: Becoming Leader. State: Replica: bc88b6ae2fd34d69b4d17083d79641e1, State: Running, Role: LEADER
I20260808 02:13:19.260022 26941 heartbeater.cc:503] Master 127.18.184.188:35335 was elected leader, sending a full tablet report...
I20260808 02:13:19.260056 27198 consensus_queue.cc:237] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } }
I20260808 02:13:19.260599 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.260311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.130:40575)
0808 02:13:19.260415 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.260513 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 02:13:19.262720 26588 catalog_manager.cc:5495] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc88b6ae2fd34d69b4d17083d79641e1 (127.18.184.130). New cstate: current_term: 1 leader_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc88b6ae2fd34d69b4d17083d79641e1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 } health_report { overall_health: UNKNOWN } } }
W20260808 02:13:19.268599 26796 tablet.cc:2369] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:19.284485 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.280510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53372 (local address 127.18.184.130:41985)
0808 02:13:19.280763 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:19.280770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:19.280787 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:13:19.280901 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:19.280903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:19.280967 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:19.281091 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:19.281463 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.282089 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.283068 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.283298 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.283382 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.283629 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:19.284115 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:19.284205 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:19.284315 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 02:13:19.284369 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:13:19.287585 27040 raft_consensus.cc:1261] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998 [term 1 FOLLOWER]: Refusing update from remote peer bc88b6ae2fd34d69b4d17083d79641e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:19.287899 26746 raft_consensus.cc:1261] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575 [term 1 FOLLOWER]: Refusing update from remote peer bc88b6ae2fd34d69b4d17083d79641e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:19.288210 27198 consensus_queue.cc:1035] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce6ceb70486646e3a3e812c025e35998" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 44559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:19.288573 27203 consensus_queue.cc:1035] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "236b94a7ada74cc28c42bf86c2db8575" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:19.289456 27090 heartbeater.cc:503] Master 127.18.184.188:35335 was elected leader, sending a full tablet report...
I20260808 02:13:19.290045 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.289722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.131:44271)
0808 02:13:19.289899 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.289959 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260808 02:13:19.291520 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.291195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.129:55057)
0808 02:13:19.291373 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.291436 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260808 02:13:19.294634 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.294256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.131:42887)
0808 02:13:19.294493 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.294559 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260808 02:13:19.295980 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.295722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.130:37655)
0808 02:13:19.295838 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.295900 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 02:13:19.296447 27206 mvcc.cc:205] Tried to move back new op lower bound from 7316091696276500480 to 7316091696172400640. Current Snapshot: MvccSnapshot[applied={T|T < 7316091696276500480 or (T in {7316091696276500480})}]
I20260808 02:13:19.297091 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.296860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.130:37733)
0808 02:13:19.296964 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.297028 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 02:13:19.298273 27207 mvcc.cc:205] Tried to move back new op lower bound from 7316091696276500480 to 7316091696172400640. Current Snapshot: MvccSnapshot[applied={T|T < 7316091696276500480 or (T in {7316091696276500480})}]
I20260808 02:13:19.300994 26590 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:19.301190 26590 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:19.301370 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.300997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43577 (local address 127.18.184.129:38167)
0808 02:13:19.301217 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.301289 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260808 02:13:19.305727 26590 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6728af60b1d24132b3fe6e66c733de88: Sending DeleteTablet for 3 replicas of tablet b598cc581fc64de0a5ac008b565efb00
I20260808 02:13:19.306433 26873 tablet_service.cc:1514] Processing DeleteTablet for tablet b598cc581fc64de0a5ac008b565efb00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:19 UTC) from {username='oryx', principal='oryx/127.18.184.188@KRBTEST.COM'} at 127.0.0.1:53362
I20260808 02:13:19.306484 27018 tablet_service.cc:1514] Processing DeleteTablet for tablet b598cc581fc64de0a5ac008b565efb00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:19 UTC) from {username='oryx', principal='oryx/127.18.184.188@KRBTEST.COM'} at 127.0.0.1:35278
I20260808 02:13:19.306790 26721 tablet_service.cc:1514] Processing DeleteTablet for tablet b598cc581fc64de0a5ac008b565efb00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:19 UTC) from {username='oryx', principal='oryx/127.18.184.188@KRBTEST.COM'} at 127.0.0.1:41304
I20260808 02:13:19.307124 26720 tablet_service.cc:1514] Processing DeleteTablet for tablet b598cc581fc64de0a5ac008b565efb00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:19 UTC) from {username='oryx', principal='oryx/127.18.184.188@KRBTEST.COM'} at 127.0.0.1:41304
I20260808 02:13:19.307246 27218 tablet_replica.cc:330] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575: stopping tablet replica
I20260808 02:13:19.307425 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 26551
I20260808 02:13:19.307435 27218 raft_consensus.cc:2229] T b598cc581fc64de0a5ac008b565efb00 P 236b94a7ada74cc28c42bf86c2db8575 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:19.317875 27216 tablet_replica.cc:330] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1: stopping tablet replica
I20260808 02:13:19.318097 27216 raft_consensus.cc:2229] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:19.318526 27216 raft_consensus.cc:2258] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:19.319090 27217 tablet_replica.cc:330] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998: stopping tablet replica
I20260808 02:13:19.319269 27217 raft_consensus.cc:2229] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:19.319434 27217 raft_consensus.cc:2258] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:19.319659 27216 ts_tablet_manager.cc:1905] T b598cc581fc64de0a5ac008b565efb00 P bc88b6ae2fd34d69b4d17083d79641e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:19.320333 27217 ts_tablet_manager.cc:1905] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 02:13:19.322669 27188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:19.322102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:33407 (local address 127.0.0.1:41318)
0808 02:13:19.322311 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.322327 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:19.322422 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:19.322538 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:33407: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":130}
I20260808 02:13:19.322872 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 26802
W20260808 02:13:19.323027 26582 catalog_manager.cc:4584] TS 236b94a7ada74cc28c42bf86c2db8575 (127.18.184.129:33407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b598cc581fc64de0a5ac008b565efb00: Network error: Client connection negotiation failed: client connection to 127.18.184.129:33407: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260808 02:13:19.327142 27217 ts_tablet_manager.cc:1918] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:19.327252 27217 log.cc:1196] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/b598cc581fc64de0a5ac008b565efb00
I20260808 02:13:19.327565 27217 ts_tablet_manager.cc:1939] T b598cc581fc64de0a5ac008b565efb00 P ce6ceb70486646e3a3e812c025e35998: Deleting consensus metadata
I20260808 02:13:19.328382 26582 catalog_manager.cc:4841] TS ce6ceb70486646e3a3e812c025e35998 (127.18.184.131:44559): tablet b598cc581fc64de0a5ac008b565efb00 (table test-table [id=dd4e70d1cc5a42828b69bf55e51fd6f6]) successfully deleted
I20260808 02:13:19.333580 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 26948
I20260808 02:13:19.333905 27188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.333562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:41985 (local address 127.0.0.1:53388)
0808 02:13:19.333718 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:19.333797 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.130:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 02:13:19.341409 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 26237
I20260808 02:13:19.349321 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 26350
2026-08-08T02:13:19Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5513 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:19 dist-test-slave-9gcw krb5kdc[27221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:19 dist-test-slave-9gcw krb5kdc[27221](info): set up 2 sockets
Aug 08 02:13:19 dist-test-slave-9gcw krb5kdc[27221](info): commencing operation
krb5kdc: starting...
W20260808 02:13:20.521044 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:13:20 dist-test-slave-9gcw krb5kdc[27221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.190.keytab.
Entry for principal oryx/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.190.keytab.
I20260808 02:13:20.664311 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:43385
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.190.keytab
--principal=oryx/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:43385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:20.783170 27235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:20.783528 27235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:20.783607 27235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:20.788278 27235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:20.788422 27235 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:13:20.788463 27235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:20.788511 27235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:20.788574 27235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:20.788615 27235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:20.794516 27235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:43385
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.190.keytab
--principal=oryx/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:43385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:20.795795 27235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:20.796900 27235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:20.805961 27242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.805943 27246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.805990 27241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:20.806113 27235 server_base.cc:1034] running on GCE node
I20260808 02:13:20.806866 27235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:20.808068 27235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:20.809224 27235 hybrid_clock.cc:648] HybridClock initialized: now 1786155200809205 us; error 24 us; skew 500 ppm
Aug 08 02:13:20 dist-test-slave-9gcw krb5kdc[27221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:20.812781 27235 init.cc:377] Logged in from keytab as oryx/127.18.184.190@KRBTEST.COM (short username oryx)
I20260808 02:13:20.814358 27235 webserver.cc:460] Webserver started at http://127.18.184.190:40835/ using document root <none> and password file <none>
I20260808 02:13:20.815023 27235 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:20.815137 27235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:20.815467 27235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:20.817981 27235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "4710f8ac53bc40b4930d98c032d04727"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-9gcw"
server_key: "a5c12c2599ca6930672c47bd8095fc06"
server_key_iv: "d067cc59622dd6e725a41c817c3778d1"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.818620 27235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "4710f8ac53bc40b4930d98c032d04727"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-9gcw"
server_key: "a5c12c2599ca6930672c47bd8095fc06"
server_key_iv: "d067cc59622dd6e725a41c817c3778d1"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.822815 27235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:13:20.825517 27254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.826959 27235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:13:20.827183 27235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "4710f8ac53bc40b4930d98c032d04727"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-9gcw"
server_key: "a5c12c2599ca6930672c47bd8095fc06"
server_key_iv: "d067cc59622dd6e725a41c817c3778d1"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.827324 27235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:20.843168 27235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:20.846626 27235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:20.846877 27235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:20.855975 27235 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:43385
I20260808 02:13:20.856006 27320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:43385 every 8 connection(s)
I20260808 02:13:20.857028 27235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:20.859988 27321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:20.862448 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 27235
I20260808 02:13:20.862584 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:20.862905 19170 external_mini_cluster.cc:1428] Setting key 8feb060fb3e0431a4d066d97aabfd62c
I20260808 02:13:20.866438 27321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727: Bootstrap starting.
I20260808 02:13:20.868747 27321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:20.869611 27321 log.cc:824] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:20.872308 27321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727: No bootstrap required, opened a new log
Aug 08 02:13:20 dist-test-slave-9gcw krb5kdc[27221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
I20260808 02:13:20.877637 27321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4710f8ac53bc40b4930d98c032d04727" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43385 } }
I20260808 02:13:20.878036 27321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:20.878115 27321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4710f8ac53bc40b4930d98c032d04727, State: Initialized, Role: FOLLOWER
I20260808 02:13:20.878664 27321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4710f8ac53bc40b4930d98c032d04727" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43385 } }
I20260808 02:13:20.878839 27321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:20.878916 27321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:20.879024 27321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:20.879355 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.864491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60766 (local address 127.18.184.190:43385)
0808 02:13:20.864950 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.864965 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.865002 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:13:20.865707 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.865724 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.866066 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.866434 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.867270 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.868285 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.869276 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.869812 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.869977 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.873064 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.873072 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.873097 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.873179 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.875453 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.875956 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.875963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.875969 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.876181 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.876568 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.876572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.876574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.877296 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.877517 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.877792 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.878159 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":149,"threads_started":1}
I20260808 02:13:20.880374 27321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4710f8ac53bc40b4930d98c032d04727" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43385 } }
I20260808 02:13:20.880776 27321 leader_election.cc:302] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4710f8ac53bc40b4930d98c032d04727; no voters:
I20260808 02:13:20.881198 27321 leader_election.cc:288] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:20.881284 27327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:20.881529 27327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [term 1 LEADER]: Becoming Leader. State: Replica: 4710f8ac53bc40b4930d98c032d04727, State: Running, Role: LEADER
I20260808 02:13:20.881929 27327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4710f8ac53bc40b4930d98c032d04727" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43385 } }
I20260808 02:13:20.882587 27321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:20.883749 27329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4710f8ac53bc40b4930d98c032d04727. Latest consensus state: current_term: 1 leader_uuid: "4710f8ac53bc40b4930d98c032d04727" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4710f8ac53bc40b4930d98c032d04727" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43385 } } }
I20260808 02:13:20.883900 27329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:20.884294 27336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:20.884330 27328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4710f8ac53bc40b4930d98c032d04727" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4710f8ac53bc40b4930d98c032d04727" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43385 } } }
I20260808 02:13:20.884459 27328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:20.887894 27336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:20.893986 27336 catalog_manager.cc:1347] Generated new cluster ID: 195e8c082a0645e19893e72ac0fb5681
I20260808 02:13:20.894078 27336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:20.904567 27336 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:20.905993 27336 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:20.920843 27336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727: Generated new TSK 0
I20260808 02:13:20.921715 27336 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab.
Entry for principal oryx/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab.
I20260808 02:13:20.968452 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:43385
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab
--principal=oryx/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:21.095245 27349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:21.095535 27349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:21.095582 27349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:21.099752 27349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:21.099824 27349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:21.099864 27349 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:13:21.099897 27349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:21.105247 27349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab
--principal=oryx/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.184.190:43385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:21.106371 27349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:21.107414 27349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:21.117278 27357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:21.117269 27354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:21.117265 27355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:21.117416 27349 server_base.cc:1034] running on GCE node
I20260808 02:13:21.118052 27349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:21.118755 27349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:21.119936 27349 hybrid_clock.cc:648] HybridClock initialized: now 1786155201119899 us; error 48 us; skew 500 ppm
Aug 08 02:13:21 dist-test-slave-9gcw krb5kdc[27221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:21.123426 27349 init.cc:377] Logged in from keytab as oryx/127.18.184.129@KRBTEST.COM (short username oryx)
I20260808 02:13:21.124411 27349 webserver.cc:460] Webserver started at http://127.18.184.129:35849/ using document root <none> and password file <none>
I20260808 02:13:21.124996 27349 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:21.125066 27349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:21.125324 27349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:21.127583 27349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "fa52b3cc791a439b88acb9dc9a3b7766"
format_stamp: "Formatted at 2026-08-08 02:13:21 on dist-test-slave-9gcw"
server_key: "de28a3aa84bf808c0ff41adb7dc131cd"
server_key_iv: "57680ff28248651fe2cc40017f18a229"
server_key_version: "encryptionkey@0"
I20260808 02:13:21.128188 27349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "fa52b3cc791a439b88acb9dc9a3b7766"
format_stamp: "Formatted at 2026-08-08 02:13:21 on dist-test-slave-9gcw"
server_key: "de28a3aa84bf808c0ff41adb7dc131cd"
server_key_iv: "57680ff28248651fe2cc40017f18a229"
server_key_version: "encryptionkey@0"
I20260808 02:13:21.133334 27349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 02:13:21.136181 27365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.137756 27349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:21.137907 27349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "fa52b3cc791a439b88acb9dc9a3b7766"
format_stamp: "Formatted at 2026-08-08 02:13:21 on dist-test-slave-9gcw"
server_key: "de28a3aa84bf808c0ff41adb7dc131cd"
server_key_iv: "57680ff28248651fe2cc40017f18a229"
server_key_version: "encryptionkey@0"
I20260808 02:13:21.138037 27349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:21.152233 27349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:21.155658 27349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:21.155915 27349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:21.157770 27349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:21.157860 27349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.157927 27349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:21.157960 27349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:13:21 dist-test-slave-9gcw krb5kdc[27221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
I20260808 02:13:21.170557 27349 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:33317
I20260808 02:13:21.170819 27488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:33317 every 8 connection(s)
I20260808 02:13:21.171765 27349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:21.173745 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.158475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:34553 (local address 127.18.184.190:43385)
0808 02:13:21.158619 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.158625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.159910 (+ 1285us) server_negotiation.cc:407] Connection header received
0808 02:13:21.161320 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.161324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.161392 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.161458 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.162971 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.163654 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.164899 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.165201 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.165324 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.169492 (+ 4168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.169496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.169499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.169537 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.171286 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.171988 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.171991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.171993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.172108 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.172567 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.172571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.172573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.172807 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.172891 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.173496 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.173620 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:13:21.174635 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.159140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43385 (local address 127.18.184.129:34553)
0808 02:13:21.159715 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.159776 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.161004 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.161891 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.161898 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.162231 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.162806 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.162821 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.163791 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.163799 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.164762 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.164772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.165372 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.165376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.165548 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.166212 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.166228 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.169269 (+ 3041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.171421 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.171428 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.171443 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.171844 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.172229 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.172232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.172234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.172439 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.172931 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.172945 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.173269 (+ 324us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.173447 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.173732 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":143,"threads_started":1}
I20260808 02:13:21.175999 27490 heartbeater.cc:347] Connected to a master server at 127.18.184.190:43385
I20260808 02:13:21.176164 27490 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:21.176667 27490 heartbeater.cc:511] Master 127.18.184.190:43385 requested a full tablet report, sending...
I20260808 02:13:21.177814 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 27349
I20260808 02:13:21.177944 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:21.178282 19170 external_mini_cluster.cc:1428] Setting key f4028980ae95aaa625de30f157eb1be7
I20260808 02:13:21.178471 27270 ts_manager.cc:194] Registered new tserver with Master: fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129:33317)
I20260808 02:13:21.179879 27270 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:34553
I20260808 02:13:21.189100 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.178586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:44021 (local address 127.18.184.190:43385)
0808 02:13:21.178767 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.178771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.179173 (+ 402us) server_negotiation.cc:407] Connection header received
0808 02:13:21.179347 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.179351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.179403 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.179479 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.180184 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.180768 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.183649 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.183883 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.183975 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.185658 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.185661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.185663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.185687 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.187228 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.187776 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.187779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.187780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.187871 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.188231 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.188234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.188235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.188421 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.188510 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.188859 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.188966 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 02:13:21.189282 27496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.178539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43385 (local address 127.18.184.129:44021)
0808 02:13:21.179077 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.179091 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.179224 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.179636 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.179639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.179901 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.180044 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.180057 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.182396 (+ 2339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.182402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.183431 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.183446 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.184010 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.184016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.184150 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.184853 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.184875 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.185462 (+ 587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.187353 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.187359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.187363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.187631 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.187986 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.187991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.187994 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.188107 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.188537 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.188543 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.188650 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.188780 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.188957 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":359,"threads_started":1}
WARNING: no policy specified for oryx/127.18.184.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab.
Entry for principal oryx/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab.
I20260808 02:13:21.212389 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:43385
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab
--principal=oryx/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:21.327812 27499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:21.328133 27499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:21.328223 27499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:21.332528 27499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:21.332620 27499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:21.332692 27499 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:13:21.332772 27499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:21.338260 27499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab
--principal=oryx/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.184.190:43385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:21.339428 27499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:21.340503 27499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:21.350340 27505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:21.350332 27507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:21.350347 27504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:21.351230 27499 server_base.cc:1034] running on GCE node
I20260808 02:13:21.351596 27499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:21.352181 27499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:21.353350 27499 hybrid_clock.cc:648] HybridClock initialized: now 1786155201353317 us; error 47 us; skew 500 ppm
Aug 08 02:13:21 dist-test-slave-9gcw krb5kdc[27221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:21.356261 27499 init.cc:377] Logged in from keytab as oryx/127.18.184.130@KRBTEST.COM (short username oryx)
I20260808 02:13:21.357298 27499 webserver.cc:460] Webserver started at http://127.18.184.130:39123/ using document root <none> and password file <none>
I20260808 02:13:21.357928 27499 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:21.357990 27499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:21.358215 27499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:21.360566 27499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "471fb1b38509487facacea4e2cbd0bb1"
format_stamp: "Formatted at 2026-08-08 02:13:21 on dist-test-slave-9gcw"
server_key: "1e5187695867224eab6288d8365932a7"
server_key_iv: "3ceedf82baa0acf4d944983f34941d4d"
server_key_version: "encryptionkey@0"
I20260808 02:13:21.361261 27499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "471fb1b38509487facacea4e2cbd0bb1"
format_stamp: "Formatted at 2026-08-08 02:13:21 on dist-test-slave-9gcw"
server_key: "1e5187695867224eab6288d8365932a7"
server_key_iv: "3ceedf82baa0acf4d944983f34941d4d"
server_key_version: "encryptionkey@0"
I20260808 02:13:21.365276 27499 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:13:21.367966 27514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.369520 27499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:21.369686 27499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "471fb1b38509487facacea4e2cbd0bb1"
format_stamp: "Formatted at 2026-08-08 02:13:21 on dist-test-slave-9gcw"
server_key: "1e5187695867224eab6288d8365932a7"
server_key_iv: "3ceedf82baa0acf4d944983f34941d4d"
server_key_version: "encryptionkey@0"
I20260808 02:13:21.369815 27499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:21.402490 27499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:21.406072 27499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:21.406315 27499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:21.407914 27499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:21.407998 27499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.408075 27499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:21.408119 27499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:13:21 dist-test-slave-9gcw krb5kdc[27221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
I20260808 02:13:21.419878 27499 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:37679
I20260808 02:13:21.419975 27635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:37679 every 8 connection(s)
I20260808 02:13:21.421026 27499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:21.424561 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.409021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:38441 (local address 127.18.184.190:43385)
0808 02:13:21.409191 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.409199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.410510 (+ 1311us) server_negotiation.cc:407] Connection header received
0808 02:13:21.411954 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.411959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.412029 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.412134 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.414016 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.414573 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.416624 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.416903 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.417008 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.420328 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.420333 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.420337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.420377 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.422320 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.423110 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.423113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.423114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.423197 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.423550 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.423552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.423553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.423736 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.423809 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.424323 (+ 514us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.424438 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:13:21.425643 27524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.409647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43385 (local address 127.18.184.130:38441)
0808 02:13:21.410321 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.410374 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.411687 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.412368 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.412377 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.412929 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.413807 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.413845 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.414692 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.414697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.416433 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.416447 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.417113 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.417119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.417380 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.418232 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.418252 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.420072 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.422452 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.422459 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.422474 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.422993 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.423294 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.423296 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.423298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.423451 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.423833 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.423839 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.424095 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.424260 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.424548 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":170,"threads_started":1}
I20260808 02:13:21.427068 27636 heartbeater.cc:347] Connected to a master server at 127.18.184.190:43385
I20260808 02:13:21.427237 27636 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:21.427959 27636 heartbeater.cc:511] Master 127.18.184.190:43385 requested a full tablet report, sending...
I20260808 02:13:21.429265 27271 ts_manager.cc:194] Registered new tserver with Master: 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130:37679)
I20260808 02:13:21.429965 27271 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:38441
I20260808 02:13:21.430711 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 27499
I20260808 02:13:21.430841 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:21.431231 19170 external_mini_cluster.cc:1428] Setting key 347bad43724d08648148a2f21c73188d
I20260808 02:13:21.438988 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.429916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:58117 (local address 127.18.184.190:43385)
0808 02:13:21.430071 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.430076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.430284 (+ 208us) server_negotiation.cc:407] Connection header received
0808 02:13:21.430441 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.430445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.430499 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.430591 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.431403 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.432249 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.433395 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.433781 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.433915 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.435266 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.435270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.435273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.435309 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.436796 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.437419 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.437423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.437425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.437520 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.437946 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.437950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.437952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.438230 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.438360 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.438660 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.438794 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:13:21.439002 27642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.429824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43385 (local address 127.18.184.130:58117)
0808 02:13:21.430183 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.430199 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.430310 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.430721 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.430724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.431010 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.431209 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.431220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.432441 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.432448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.433264 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.433275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.433941 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.433944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.434032 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.434612 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.434624 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.435071 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.436929 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.436933 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.436936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.437300 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.437628 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.437631 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.437634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.437821 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.438347 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.438351 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.438471 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.438596 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.438797 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":1,"thread_start_us":150,"threads_started":1}
WARNING: no policy specified for oryx/127.18.184.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab.
Entry for principal oryx/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab.
I20260808 02:13:21.465500 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:43385
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab
--principal=oryx/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:21.583289 27645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:21.583671 27645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:21.583724 27645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:21.587921 27645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:21.587989 27645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:21.588029 27645 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:13:21.588063 27645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:21.593521 27645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab
--principal=oryx/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.184.190:43385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:21.594640 27645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:21.595700 27645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:21.606235 27653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:21.606235 27651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:21.606606 27645 server_base.cc:1034] running on GCE node
W20260808 02:13:21.606266 27650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:21.607203 27645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:21.607873 27645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:21.609061 27645 hybrid_clock.cc:648] HybridClock initialized: now 1786155201609030 us; error 41 us; skew 500 ppm
Aug 08 02:13:21 dist-test-slave-9gcw krb5kdc[27221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:21.612298 27645 init.cc:377] Logged in from keytab as oryx/127.18.184.131@KRBTEST.COM (short username oryx)
I20260808 02:13:21.613428 27645 webserver.cc:460] Webserver started at http://127.18.184.131:34219/ using document root <none> and password file <none>
I20260808 02:13:21.614084 27645 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:21.614197 27645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:21.614473 27645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:21.616972 27645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "ddc2a42726d147e8b34e8dd90cadc66c"
format_stamp: "Formatted at 2026-08-08 02:13:21 on dist-test-slave-9gcw"
server_key: "830e833f72d774304f8ac7a0f4524104"
server_key_iv: "d9f390f184f7c5aa9e7c47faaae8abc4"
server_key_version: "encryptionkey@0"
I20260808 02:13:21.617774 27645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "ddc2a42726d147e8b34e8dd90cadc66c"
format_stamp: "Formatted at 2026-08-08 02:13:21 on dist-test-slave-9gcw"
server_key: "830e833f72d774304f8ac7a0f4524104"
server_key_iv: "d9f390f184f7c5aa9e7c47faaae8abc4"
server_key_version: "encryptionkey@0"
I20260808 02:13:21.622500 27645 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:13:21.625610 27660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.627125 27645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:13:21.627359 27645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "ddc2a42726d147e8b34e8dd90cadc66c"
format_stamp: "Formatted at 2026-08-08 02:13:21 on dist-test-slave-9gcw"
server_key: "830e833f72d774304f8ac7a0f4524104"
server_key_iv: "d9f390f184f7c5aa9e7c47faaae8abc4"
server_key_version: "encryptionkey@0"
I20260808 02:13:21.627518 27645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:21.640552 27645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:21.644157 27645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:21.644385 27645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:21.645991 27645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:21.646044 27645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.646085 27645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:21.646140 27645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:13:21 dist-test-slave-9gcw krb5kdc[27221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
I20260808 02:13:21.657887 27645 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:38895
I20260808 02:13:21.657969 27780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:38895 every 8 connection(s)
I20260808 02:13:21.659116 27645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:21.661684 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.646845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:33955 (local address 127.18.184.190:43385)
0808 02:13:21.646981 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.646984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.648106 (+ 1122us) server_negotiation.cc:407] Connection header received
0808 02:13:21.649291 (+ 1185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.649297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.649384 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.649458 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.651420 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.652321 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.654214 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.654548 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.654664 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.657908 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.657913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.657916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.657949 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.659618 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.660239 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.660243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.660244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.660335 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.660613 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.660616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.660617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.660798 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.660876 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.661452 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.661561 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:13:21.662529 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.647410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43385 (local address 127.18.184.131:33955)
0808 02:13:21.647952 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.647996 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.648993 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.649979 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.649987 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.650440 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.651232 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.651253 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.652470 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.652475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.654030 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.654041 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.654721 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.654724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.654867 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.655606 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.655626 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.657688 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.659755 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.659760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.659778 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.660123 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.660400 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.660402 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.660404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.660529 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.660888 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.660893 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.661114 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.661381 (+ 267us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.661633 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":162,"threads_started":1}
I20260808 02:13:21.663684 27781 heartbeater.cc:347] Connected to a master server at 127.18.184.190:43385
I20260808 02:13:21.663846 27781 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:21.664121 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 27645
I20260808 02:13:21.664259 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:21.664395 27781 heartbeater.cc:511] Master 127.18.184.190:43385 requested a full tablet report, sending...
I20260808 02:13:21.664647 19170 external_mini_cluster.cc:1428] Setting key a924a91558fd5e1a65a0ed8ade786b2e
I20260808 02:13:21.665674 27270 ts_manager.cc:194] Registered new tserver with Master: ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895)
I20260808 02:13:21.666476 27270 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:33955
I20260808 02:13:21.667382 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:13:21.673940 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.666287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:40951 (local address 127.18.184.190:43385)
0808 02:13:21.666420 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.666425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.666610 (+ 185us) server_negotiation.cc:407] Connection header received
0808 02:13:21.666765 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.666769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.666812 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.666872 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.667482 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.667982 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.668823 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.669043 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.669152 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.670502 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.670504 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.670506 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.670528 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.672060 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.672502 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.672506 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.672508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.672591 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.672886 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.672890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.672892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.673212 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.673348 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.673696 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.673807 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:13:21.673951 27787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.666214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43385 (local address 127.18.184.131:40951)
0808 02:13:21.666516 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.666548 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.666651 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.667007 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.667009 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.667251 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.667362 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.667369 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.668111 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.668114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.668690 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.668699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.669177 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.669181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.669284 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.669862 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.669882 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.670358 (+ 476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.672186 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.672189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.672192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.672398 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.672694 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.672699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.672701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.672792 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.673336 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.673340 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.673463 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.673571 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.673748 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":11,"thread_start_us":97,"threads_started":1}
Aug 08 02:13:21 dist-test-slave-9gcw krb5kdc[27221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:13:21 dist-test-slave-9gcw krb5kdc[27221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:21 dist-test-slave-9gcw krb5kdc[27221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:13:21.695466 27324 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:21.689120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60778 (local address 127.18.184.190:43385)
0808 02:13:21.689305 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.689310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.689323 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:13:21.689368 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.689371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.689410 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.689490 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.690218 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.690767 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.691606 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.691815 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.691899 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.695290 (+ 3391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60778: BlockingRecv error: recv got EOF from 127.0.0.1:60778 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 08 02:13:21 dist-test-slave-9gcw krb5kdc[27221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
I20260808 02:13:21.705313 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.696864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60790 (local address 127.18.184.190:43385)
0808 02:13:21.696983 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.696986 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.697094 (+ 108us) server_negotiation.cc:407] Connection header received
0808 02:13:21.697287 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.697289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.697336 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.697405 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.697967 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.698461 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.699376 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.699580 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.699653 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.702204 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.702207 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.702209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.702235 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.703713 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.704131 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.704133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.704135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.704208 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.704469 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.704471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.704472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.704646 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.704750 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.705020 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.705118 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 02:13:21.708544 27271 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:21.711798 27271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:21.728277 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.721294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33136 (local address 127.18.184.130:37679)
0808 02:13:21.722475 (+ 1181us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.722487 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.722510 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:13:21.722590 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.722597 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.722793 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.723087 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:21.723324 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.724547 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.726530 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.727385 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.727545 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.727692 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.728010 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.728115 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1073,"thread_start_us":141,"threads_started":1}
I20260808 02:13:21.728252 27805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.721095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:37679 (local address 127.0.0.1:33136)
0808 02:13:21.721508 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.721562 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.721879 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.722943 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.722946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.722957 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:21.723205 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.723214 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.724703 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.724706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.726416 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.726428 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.727524 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.727526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.727627 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.727790 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.727914 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.728078 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":172,"thread_start_us":121,"threads_started":1}
I20260808 02:13:21.729094 27808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.721530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:38895 (local address 127.0.0.1:35570)
0808 02:13:21.722123 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.722140 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.722235 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.722906 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.722914 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.722943 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:21.723150 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.723159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.724691 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.724696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.726335 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.726345 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.728654 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.728657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.728774 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.728802 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.728878 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.728953 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":339,"spinlock_wait_cycles":1920,"thread_start_us":104,"threads_started":1}
I20260808 02:13:21.729593 27809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.721857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.18.184.131:38895)
0808 02:13:21.722178 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.722192 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.722217 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:13:21.722403 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.722409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.722645 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.722801 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:21.723324 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.724555 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.726473 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.727387 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.727544 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.727692 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.729328 (+ 1636us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.729439 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":122,"threads_started":1}
I20260808 02:13:21.729700 27811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.722044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57018 (local address 127.18.184.129:33317)
0808 02:13:21.722445 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.722451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.722768 (+ 317us) server_negotiation.cc:407] Connection header received
0808 02:13:21.722844 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.722849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.723016 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.723180 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:21.723545 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.724479 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.727766 (+ 3287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.728548 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.728673 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.728845 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.729449 (+ 604us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.729574 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":106,"threads_started":1}
I20260808 02:13:21.729358 27807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.721828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:33317 (local address 127.0.0.1:57018)
0808 02:13:21.722569 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.722581 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.722670 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.723322 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.723325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.723344 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:21.723426 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.723430 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.724624 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.724630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.726416 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.726428 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.728912 (+ 2484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.728915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.729033 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.729062 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.729131 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.729238 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":150,"thread_start_us":94,"threads_started":1}
I20260808 02:13:21.731778 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 309fcf3915044b33ba5ab1a4be7d243e (DEFAULT_TABLE table=test-table [id=8cd00d44679a495eb6eefa2f7216a0ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:21.732357 27418 tablet_service.cc:1467] Processing CreateTablet for tablet 309fcf3915044b33ba5ab1a4be7d243e (DEFAULT_TABLE table=test-table [id=8cd00d44679a495eb6eefa2f7216a0ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:21.733099 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 309fcf3915044b33ba5ab1a4be7d243e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:21.733455 27418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 309fcf3915044b33ba5ab1a4be7d243e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:21.735960 27714 tablet_service.cc:1467] Processing CreateTablet for tablet 309fcf3915044b33ba5ab1a4be7d243e (DEFAULT_TABLE table=test-table [id=8cd00d44679a495eb6eefa2f7216a0ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:21.737250 27714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 309fcf3915044b33ba5ab1a4be7d243e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:21.739920 27812 tablet_bootstrap.cc:492] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766: Bootstrap starting.
I20260808 02:13:21.742147 27813 tablet_bootstrap.cc:492] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1: Bootstrap starting.
I20260808 02:13:21.742372 27812 tablet_bootstrap.cc:654] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:21.743356 27812 log.cc:824] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:21.744889 27813 tablet_bootstrap.cc:654] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:21.745625 27812 tablet_bootstrap.cc:492] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766: No bootstrap required, opened a new log
I20260808 02:13:21.745848 27812 ts_tablet_manager.cc:1397] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:13:21.746061 27813 log.cc:824] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:21.746757 27815 tablet_bootstrap.cc:492] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c: Bootstrap starting.
I20260808 02:13:21.748971 27813 tablet_bootstrap.cc:492] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1: No bootstrap required, opened a new log
I20260808 02:13:21.749065 27815 tablet_bootstrap.cc:654] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:21.749230 27813 ts_tablet_manager.cc:1397] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:13:21.749993 27815 log.cc:824] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:21.749866 27812 raft_consensus.cc:348] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:21.750200 27812 raft_consensus.cc:374] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:21.750270 27812 raft_consensus.cc:729] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa52b3cc791a439b88acb9dc9a3b7766, State: Initialized, Role: FOLLOWER
I20260808 02:13:21.750825 27812 consensus_queue.cc:260] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:21.751526 27490 heartbeater.cc:503] Master 127.18.184.190:43385 was elected leader, sending a full tablet report...
I20260808 02:13:21.751921 27812 ts_tablet_manager.cc:1428] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:13:21.753537 27813 raft_consensus.cc:348] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:21.753857 27813 raft_consensus.cc:374] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:21.753930 27813 raft_consensus.cc:729] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 471fb1b38509487facacea4e2cbd0bb1, State: Initialized, Role: FOLLOWER
I20260808 02:13:21.754298 27815 tablet_bootstrap.cc:492] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c: No bootstrap required, opened a new log
I20260808 02:13:21.754531 27815 ts_tablet_manager.cc:1397] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 02:13:21.754459 27813 consensus_queue.cc:260] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:21.755239 27636 heartbeater.cc:503] Master 127.18.184.190:43385 was elected leader, sending a full tablet report...
I20260808 02:13:21.756381 27813 ts_tablet_manager.cc:1428] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:13:21.758636 27815 raft_consensus.cc:348] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:21.758934 27815 raft_consensus.cc:374] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:21.758998 27815 raft_consensus.cc:729] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddc2a42726d147e8b34e8dd90cadc66c, State: Initialized, Role: FOLLOWER
I20260808 02:13:21.759491 27815 consensus_queue.cc:260] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:21.760236 27781 heartbeater.cc:503] Master 127.18.184.190:43385 was elected leader, sending a full tablet report...
I20260808 02:13:21.760627 27815 ts_tablet_manager.cc:1428] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260808 02:13:21.910329 27782 tablet.cc:2369] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:13:21.922591 27637 tablet.cc:2369] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:13:21.923338 27491 tablet.cc:2369] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:22.081939 27818 raft_consensus.cc:482] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:22.082202 27818 raft_consensus.cc:504] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.083707 27818 leader_election.cc:288] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130:37679), ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895)
I20260808 02:13:22.088608 27809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.084182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:54471 (local address 127.18.184.131:38895)
0808 02:13:22.084357 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.084363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.084685 (+ 322us) server_negotiation.cc:407] Connection header received
0808 02:13:22.084790 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.084793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.084846 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.084920 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:22.085347 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.086201 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.087506 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.087918 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.088063 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.088128 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.088368 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.088451 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:13:22.088644 27822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.084018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:37679 (local address 127.18.184.129:45447)
0808 02:13:22.084490 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.084502 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:22.084637 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:22.085074 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:22.085076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:22.085091 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:22.085252 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.085260 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.086336 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.086339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.087325 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.087336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.088083 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.088087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.088241 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.088260 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 02:13:22.088358 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 02:13:22.088434 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":74,"thread_start_us":156,"threads_started":1}
I20260808 02:13:22.088630 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.084286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:45447 (local address 127.18.184.130:37679)
0808 02:13:22.084432 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.084438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.084598 (+ 160us) server_negotiation.cc:407] Connection header received
0808 02:13:22.084794 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.084797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.084857 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.084912 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:22.085411 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.086194 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.087482 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.087922 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.088076 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.088140 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.088404 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.088499 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:13:22.088656 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.084034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:38895 (local address 127.18.184.129:54471)
0808 02:13:22.084514 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.084528 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:22.084690 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:22.084969 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:22.084974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:22.084999 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:22.085194 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.085206 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.086390 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.086395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.087325 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.087336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.088063 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.088067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.088218 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.088244 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:13:22.088316 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 02:13:22.088432 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":110,"thread_start_us":123,"threads_started":1}
I20260808 02:13:22.089564 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309fcf3915044b33ba5ab1a4be7d243e" candidate_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471fb1b38509487facacea4e2cbd0bb1" is_pre_election: true
I20260808 02:13:22.089643 27735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309fcf3915044b33ba5ab1a4be7d243e" candidate_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" is_pre_election: true
I20260808 02:13:22.089932 27588 raft_consensus.cc:2454] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa52b3cc791a439b88acb9dc9a3b7766 in term 0.
I20260808 02:13:22.090001 27735 raft_consensus.cc:2454] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa52b3cc791a439b88acb9dc9a3b7766 in term 0.
I20260808 02:13:22.090612 27366 leader_election.cc:302] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 471fb1b38509487facacea4e2cbd0bb1, fa52b3cc791a439b88acb9dc9a3b7766; no voters:
I20260808 02:13:22.090983 27818 raft_consensus.cc:2781] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:22.091074 27818 raft_consensus.cc:482] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:22.091133 27818 raft_consensus.cc:3037] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:22.092299 27818 raft_consensus.cc:504] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.092751 27818 leader_election.cc:288] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [CANDIDATE]: Term 1 election: Requested vote from peers 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130:37679), ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895)
I20260808 02:13:22.093240 27735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309fcf3915044b33ba5ab1a4be7d243e" candidate_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc2a42726d147e8b34e8dd90cadc66c"
I20260808 02:13:22.093250 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309fcf3915044b33ba5ab1a4be7d243e" candidate_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471fb1b38509487facacea4e2cbd0bb1"
I20260808 02:13:22.093403 27735 raft_consensus.cc:3037] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:22.093417 27588 raft_consensus.cc:3037] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:22.094556 27588 raft_consensus.cc:2454] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa52b3cc791a439b88acb9dc9a3b7766 in term 1.
I20260808 02:13:22.094955 27366 leader_election.cc:302] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 471fb1b38509487facacea4e2cbd0bb1, fa52b3cc791a439b88acb9dc9a3b7766; no voters:
I20260808 02:13:22.095170 27818 raft_consensus.cc:2781] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:22.095180 27735 raft_consensus.cc:2454] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa52b3cc791a439b88acb9dc9a3b7766 in term 1.
I20260808 02:13:22.095531 27818 raft_consensus.cc:686] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 LEADER]: Becoming Leader. State: Replica: fa52b3cc791a439b88acb9dc9a3b7766, State: Running, Role: LEADER
I20260808 02:13:22.095984 27818 consensus_queue.cc:237] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.099845 27270 catalog_manager.cc:5495] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129). New cstate: current_term: 1 leader_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:22.183172 27811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.179323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57024 (local address 127.18.184.129:33317)
0808 02:13:22.179474 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.179479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.179495 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:13:22.179595 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.179598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.179654 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.179758 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:22.179986 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.180786 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.181771 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.181985 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.182070 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.182309 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:22.182755 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:22.182921 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.183002 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.183060 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:13:22.189462 27735 raft_consensus.cc:1261] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Refusing update from remote peer fa52b3cc791a439b88acb9dc9a3b7766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:22.189462 27588 raft_consensus.cc:1261] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Refusing update from remote peer fa52b3cc791a439b88acb9dc9a3b7766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:22.190233 27818 consensus_queue.cc:1035] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:22.190470 27824 consensus_queue.cc:1035] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:22.199146 27828 mvcc.cc:205] Tried to move back new op lower bound from 7316091708160212992 to 7316091707788701696. Current Snapshot: MvccSnapshot[applied={T|T < 7316091708160212992}]
I20260808 02:13:22.205835 27270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:22.206058 27270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:22.208899 27270 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727: Sending DeleteTablet for 3 replicas of tablet 309fcf3915044b33ba5ab1a4be7d243e
I20260808 02:13:22.209908 27418 tablet_service.cc:1514] Processing DeleteTablet for tablet 309fcf3915044b33ba5ab1a4be7d243e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:22 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:57018
I20260808 02:13:22.209877 27714 tablet_service.cc:1514] Processing DeleteTablet for tablet 309fcf3915044b33ba5ab1a4be7d243e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:22 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:35570
I20260808 02:13:22.209942 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet 309fcf3915044b33ba5ab1a4be7d243e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:22 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:33136
I20260808 02:13:22.210438 27837 tablet_replica.cc:330] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1: stopping tablet replica
I20260808 02:13:22.210680 27840 tablet_replica.cc:330] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c: stopping tablet replica
I20260808 02:13:22.210595 27270 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 02:13:22.210835 27837 raft_consensus.cc:2229] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 02:13:22.211076 27270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:22.211200 27837 raft_consensus.cc:2258] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:22.211336 27841 tablet_replica.cc:330] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766: stopping tablet replica
I20260808 02:13:22.211714 27840 raft_consensus.cc:2229] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:22.212097 27841 raft_consensus.cc:2229] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:22.212615 27841 raft_consensus.cc:2258] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:22.212988 27840 raft_consensus.cc:2258] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:22.214879 27840 ts_tablet_manager.cc:1905] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:22.215373 27841 ts_tablet_manager.cc:1905] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:22.217414 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 9fa192794de04f1da636f53f0313cace (DEFAULT_TABLE table=test-table [id=a5f545d16dd6461aaf8b4cbfdcadfc29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:22.217728 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fa192794de04f1da636f53f0313cace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:22.218233 27840 ts_tablet_manager.cc:1918] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:22.218338 27841 ts_tablet_manager.cc:1918] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:22.218444 27841 log.cc:1196] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/wal/wals/309fcf3915044b33ba5ab1a4be7d243e
I20260808 02:13:22.218449 27840 log.cc:1196] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/309fcf3915044b33ba5ab1a4be7d243e
I20260808 02:13:22.218804 27841 ts_tablet_manager.cc:1939] T 309fcf3915044b33ba5ab1a4be7d243e P fa52b3cc791a439b88acb9dc9a3b7766: Deleting consensus metadata
I20260808 02:13:22.219012 27840 ts_tablet_manager.cc:1939] T 309fcf3915044b33ba5ab1a4be7d243e P ddc2a42726d147e8b34e8dd90cadc66c: Deleting consensus metadata
I20260808 02:13:22.220332 27258 catalog_manager.cc:4841] TS ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895): tablet 309fcf3915044b33ba5ab1a4be7d243e (table test-table [id=8cd00d44679a495eb6eefa2f7216a0ac]) successfully deleted
I20260808 02:13:22.220505 27255 catalog_manager.cc:4841] TS fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129:33317): tablet 309fcf3915044b33ba5ab1a4be7d243e (table test-table [id=8cd00d44679a495eb6eefa2f7216a0ac]) successfully deleted
I20260808 02:13:22.220526 27714 tablet_service.cc:1467] Processing CreateTablet for tablet 9fa192794de04f1da636f53f0313cace (DEFAULT_TABLE table=test-table [id=a5f545d16dd6461aaf8b4cbfdcadfc29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:22.220782 27714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fa192794de04f1da636f53f0313cace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:22.220817 27813 tablet_bootstrap.cc:492] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1: Bootstrap starting.
I20260808 02:13:22.221479 27418 tablet_service.cc:1467] Processing CreateTablet for tablet 9fa192794de04f1da636f53f0313cace (DEFAULT_TABLE table=test-table [id=a5f545d16dd6461aaf8b4cbfdcadfc29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:22.221735 27418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fa192794de04f1da636f53f0313cace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:22.221907 27813 tablet_bootstrap.cc:654] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:22.223198 27815 tablet_bootstrap.cc:492] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c: Bootstrap starting.
I20260808 02:13:22.223510 27837 ts_tablet_manager.cc:1905] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:22.224201 27815 tablet_bootstrap.cc:654] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:22.224253 27812 tablet_bootstrap.cc:492] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766: Bootstrap starting.
I20260808 02:13:22.225697 27812 tablet_bootstrap.cc:654] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:22.225740 27815 tablet_bootstrap.cc:492] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c: No bootstrap required, opened a new log
I20260808 02:13:22.225845 27815 ts_tablet_manager.cc:1397] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:22.226284 27815 raft_consensus.cc:348] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.226406 27815 raft_consensus.cc:374] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:22.226439 27815 raft_consensus.cc:729] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddc2a42726d147e8b34e8dd90cadc66c, State: Initialized, Role: FOLLOWER
I20260808 02:13:22.226564 27815 consensus_queue.cc:260] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.226811 27815 ts_tablet_manager.cc:1428] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:22.227123 27812 tablet_bootstrap.cc:492] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766: No bootstrap required, opened a new log
I20260808 02:13:22.227102 27837 ts_tablet_manager.cc:1918] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:22.227213 27812 ts_tablet_manager.cc:1397] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:22.227237 27837 log.cc:1196] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/309fcf3915044b33ba5ab1a4be7d243e
I20260808 02:13:22.227656 27837 ts_tablet_manager.cc:1939] T 309fcf3915044b33ba5ab1a4be7d243e P 471fb1b38509487facacea4e2cbd0bb1: Deleting consensus metadata
I20260808 02:13:22.227882 27812 raft_consensus.cc:348] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.227998 27812 raft_consensus.cc:374] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:22.228036 27812 raft_consensus.cc:729] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa52b3cc791a439b88acb9dc9a3b7766, State: Initialized, Role: FOLLOWER
I20260808 02:13:22.228158 27812 consensus_queue.cc:260] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.228413 27812 ts_tablet_manager.cc:1428] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:22.229101 27255 catalog_manager.cc:4841] TS 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130:37679): tablet 309fcf3915044b33ba5ab1a4be7d243e (table test-table [id=8cd00d44679a495eb6eefa2f7216a0ac]) successfully deleted
I20260808 02:13:22.229990 27813 tablet_bootstrap.cc:492] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1: No bootstrap required, opened a new log
I20260808 02:13:22.230095 27813 ts_tablet_manager.cc:1397] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 02:13:22.230607 27813 raft_consensus.cc:348] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.230720 27813 raft_consensus.cc:374] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:22.230754 27813 raft_consensus.cc:729] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 471fb1b38509487facacea4e2cbd0bb1, State: Initialized, Role: FOLLOWER
I20260808 02:13:22.230918 27813 consensus_queue.cc:260] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.231241 27813 ts_tablet_manager.cc:1428] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:22.381167 27821 raft_consensus.cc:482] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:22.381382 27821 raft_consensus.cc:504] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.382866 27821 leader_election.cc:288] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129:33317), 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130:37679)
I20260808 02:13:22.388474 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.383767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:38777 (local address 127.18.184.130:37679)
0808 02:13:22.383971 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.383976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.384123 (+ 147us) server_negotiation.cc:407] Connection header received
0808 02:13:22.384243 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.384248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.384305 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.384419 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:22.384805 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.385531 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.387030 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.387627 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.387766 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.387823 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.388232 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.388326 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 02:13:22.388461 27811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.383312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:41019 (local address 127.18.184.129:33317)
0808 02:13:22.383471 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.383474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.383622 (+ 148us) server_negotiation.cc:407] Connection header received
0808 02:13:22.383868 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.383872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.383937 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.384042 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:22.384493 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.385289 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.387031 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.387647 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.387766 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.387823 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.388232 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.388333 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:13:22.388461 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.383162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:33317 (local address 127.18.184.131:41019)
0808 02:13:22.383501 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.383521 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:22.383662 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:22.384078 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:22.384107 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:22.384150 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:22.384315 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.384324 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.385411 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.385414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.386872 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.386884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.387830 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.387833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.388006 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.388058 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 02:13:22.388134 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:13:22.388231 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":82,"spinlock_wait_cycles":5376,"thread_start_us":143,"threads_started":1}
I20260808 02:13:22.388468 27846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.383672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:37679 (local address 127.18.184.131:38777)
0808 02:13:22.384022 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.384037 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:22.384146 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:22.384555 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:22.384557 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:22.384572 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:22.384681 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.384686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.385655 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.385658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.386871 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.386884 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.387827 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.387832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.388011 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.388051 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 02:13:22.388134 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 02:13:22.388231 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":84,"spinlock_wait_cycles":3840,"thread_start_us":120,"threads_started":1}
I20260808 02:13:22.389246 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fa192794de04f1da636f53f0313cace" candidate_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471fb1b38509487facacea4e2cbd0bb1" is_pre_election: true
I20260808 02:13:22.389286 27439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fa192794de04f1da636f53f0313cace" candidate_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" is_pre_election: true
I20260808 02:13:22.389469 27588 raft_consensus.cc:2454] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddc2a42726d147e8b34e8dd90cadc66c in term 0.
I20260808 02:13:22.389526 27439 raft_consensus.cc:2454] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddc2a42726d147e8b34e8dd90cadc66c in term 0.
I20260808 02:13:22.389978 27661 leader_election.cc:302] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ddc2a42726d147e8b34e8dd90cadc66c, fa52b3cc791a439b88acb9dc9a3b7766; no voters:
I20260808 02:13:22.390249 27821 raft_consensus.cc:2781] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:22.390313 27821 raft_consensus.cc:482] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:22.390350 27821 raft_consensus.cc:3037] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:22.391391 27821 raft_consensus.cc:504] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.391753 27821 leader_election.cc:288] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [CANDIDATE]: Term 1 election: Requested vote from peers fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129:33317), 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130:37679)
I20260808 02:13:22.392211 27439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fa192794de04f1da636f53f0313cace" candidate_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa52b3cc791a439b88acb9dc9a3b7766"
I20260808 02:13:22.392264 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fa192794de04f1da636f53f0313cace" candidate_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471fb1b38509487facacea4e2cbd0bb1"
I20260808 02:13:22.392362 27439 raft_consensus.cc:3037] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:22.392377 27588 raft_consensus.cc:3037] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:22.393615 27588 raft_consensus.cc:2454] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddc2a42726d147e8b34e8dd90cadc66c in term 1.
I20260808 02:13:22.393615 27439 raft_consensus.cc:2454] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddc2a42726d147e8b34e8dd90cadc66c in term 1.
I20260808 02:13:22.393910 27661 leader_election.cc:302] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 471fb1b38509487facacea4e2cbd0bb1, ddc2a42726d147e8b34e8dd90cadc66c; no voters:
I20260808 02:13:22.394147 27821 raft_consensus.cc:2781] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:22.394410 27821 raft_consensus.cc:686] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 1 LEADER]: Becoming Leader. State: Replica: ddc2a42726d147e8b34e8dd90cadc66c, State: Running, Role: LEADER
I20260808 02:13:22.394761 27821 consensus_queue.cc:237] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.397796 27269 catalog_manager.cc:5495] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c reported cstate change: term changed from 0 to 1, leader changed from <none> to ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131). New cstate: current_term: 1 leader_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } health_report { overall_health: HEALTHY } } }
Aug 08 02:13:22 dist-test-slave-9gcw krb5kdc[27221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.190@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
I20260808 02:13:22.447796 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.439090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58870 (local address 127.18.184.190:43385)
0808 02:13:22.439412 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.439421 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.439522 (+ 101us) server_negotiation.cc:407] Connection header received
0808 02:13:22.439634 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.439636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.439678 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.439765 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:22.440416 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.440949 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.441864 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.442157 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.442238 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.444550 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:22.444553 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:22.444555 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:22.444577 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:22.446200 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:22.446641 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:22.446644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:22.446645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:22.446724 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:22.446971 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:22.446974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:22.446975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:22.447159 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:22.447240 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.447520 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.447654 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":211,"thread_start_us":112,"threads_started":1}
I20260808 02:13:22.447800 27855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.439039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43385 (local address 127.0.0.1:58870)
0808 02:13:22.439432 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.439446 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:22.439550 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:22.439813 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:22.439815 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:22.440203 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:22.440307 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.440312 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.441044 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.441047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.441748 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.441758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.442259 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.442262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.442323 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.442901 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:22.442913 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:22.444416 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:22.446310 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:22.446319 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:22.446324 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:22.446554 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:22.446806 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:22.446808 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:22.446810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:22.446894 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:22.447249 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:22.447254 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:22.447367 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 02:13:22.447459 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 02:13:22.447589 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":151,"threads_started":1}
I20260808 02:13:22.452099 27269 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:58870:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:13:22.452872 27269 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:22.457628 27418 tablet_service.cc:1467] Processing CreateTablet for tablet 03c8ba77acdf4ff59e8217ce66b26a7a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=408b28144ed5496a8464fd42fe04d966]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:13:22.457705 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 03c8ba77acdf4ff59e8217ce66b26a7a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=408b28144ed5496a8464fd42fe04d966]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:13:22.457826 27714 tablet_service.cc:1467] Processing CreateTablet for tablet 03c8ba77acdf4ff59e8217ce66b26a7a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=408b28144ed5496a8464fd42fe04d966]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:13:22.458047 27418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c8ba77acdf4ff59e8217ce66b26a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:22.458110 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c8ba77acdf4ff59e8217ce66b26a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:22.458146 27714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c8ba77acdf4ff59e8217ce66b26a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:22.460307 27812 tablet_bootstrap.cc:492] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766: Bootstrap starting.
I20260808 02:13:22.460954 27815 tablet_bootstrap.cc:492] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c: Bootstrap starting.
I20260808 02:13:22.461308 27812 tablet_bootstrap.cc:654] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:22.461400 27813 tablet_bootstrap.cc:492] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: Bootstrap starting.
I20260808 02:13:22.461972 27815 tablet_bootstrap.cc:654] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:22.462433 27813 tablet_bootstrap.cc:654] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:22.462522 27812 tablet_bootstrap.cc:492] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766: No bootstrap required, opened a new log
I20260808 02:13:22.462638 27812 ts_tablet_manager.cc:1397] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 02:13:22.463116 27812 raft_consensus.cc:348] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.463255 27812 raft_consensus.cc:374] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:22.463297 27812 raft_consensus.cc:729] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa52b3cc791a439b88acb9dc9a3b7766, State: Initialized, Role: FOLLOWER
I20260808 02:13:22.463346 27815 tablet_bootstrap.cc:492] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c: No bootstrap required, opened a new log
I20260808 02:13:22.463438 27815 ts_tablet_manager.cc:1397] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:22.463460 27812 consensus_queue.cc:260] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.463768 27812 ts_tablet_manager.cc:1428] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:22.463732 27818 tablet_replica.cc:439] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } } }
I20260808 02:13:22.463927 27818 tablet_replica.cc:442] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:22.463903 27815 raft_consensus.cc:348] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.464000 27815 raft_consensus.cc:374] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:22.464031 27815 raft_consensus.cc:729] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddc2a42726d147e8b34e8dd90cadc66c, State: Initialized, Role: FOLLOWER
I20260808 02:13:22.464162 27815 consensus_queue.cc:260] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.464468 27815 ts_tablet_manager.cc:1428] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:22.464676 27821 tablet_replica.cc:439] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } } }
I20260808 02:13:22.464821 27821 tablet_replica.cc:442] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:22.464876 27813 tablet_bootstrap.cc:492] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: No bootstrap required, opened a new log
I20260808 02:13:22.464962 27813 ts_tablet_manager.cc:1397] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 02:13:22.465498 27813 raft_consensus.cc:348] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.465619 27813 raft_consensus.cc:374] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:22.465660 27813 raft_consensus.cc:729] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 471fb1b38509487facacea4e2cbd0bb1, State: Initialized, Role: FOLLOWER
I20260808 02:13:22.465791 27813 consensus_queue.cc:260] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.466053 27820 tablet_replica.cc:439] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } } }
I20260808 02:13:22.466205 27820 tablet_replica.cc:442] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:22.467063 27813 ts_tablet_manager.cc:1428] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:22.594692 27820 raft_consensus.cc:482] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:22.594899 27820 raft_consensus.cc:504] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.596302 27820 leader_election.cc:288] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129:33317), ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895)
I20260808 02:13:22.600558 27811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.596761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:52437 (local address 127.18.184.129:33317)
0808 02:13:22.596921 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.596929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.597098 (+ 169us) server_negotiation.cc:407] Connection header received
0808 02:13:22.597289 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.597292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.597351 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.597428 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:22.597767 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.598325 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.599502 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.599924 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.600056 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.600117 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.600331 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.600417 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 02:13:22.600581 27861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.596674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:38895 (local address 127.18.184.130:51927)
0808 02:13:22.597051 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.597087 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:22.597279 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:22.597592 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:22.597594 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:22.597609 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:22.597729 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.597735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.598488 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.598490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.599374 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.599383 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.600073 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.600076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.600224 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.600242 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 02:13:22.600341 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 02:13:22.600411 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":62,"thread_start_us":107,"threads_started":1}
I20260808 02:13:22.600591 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.596540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:33317 (local address 127.18.184.130:52437)
0808 02:13:22.596982 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.596999 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:22.597131 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:22.597459 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:22.597462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:22.597479 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:22.597645 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.597652 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.598441 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.598446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.599373 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.599382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.600052 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.600056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.600202 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.600222 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:13:22.600285 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 02:13:22.600377 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":31,"thread_start_us":148,"threads_started":1}
I20260808 02:13:22.600605 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.596806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:51927 (local address 127.18.184.131:38895)
0808 02:13:22.597045 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.597051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.597215 (+ 164us) server_negotiation.cc:407] Connection header received
0808 02:13:22.597395 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.597397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.597469 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.597561 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:22.597848 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.598350 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.599504 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.599932 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.600068 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.600129 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.600382 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.600469 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":66,"threads_started":1}
I20260808 02:13:22.601092 27439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c8ba77acdf4ff59e8217ce66b26a7a" candidate_uuid: "471fb1b38509487facacea4e2cbd0bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" is_pre_election: true
I20260808 02:13:22.601092 27735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c8ba77acdf4ff59e8217ce66b26a7a" candidate_uuid: "471fb1b38509487facacea4e2cbd0bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" is_pre_election: true
I20260808 02:13:22.601284 27735 raft_consensus.cc:2454] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 471fb1b38509487facacea4e2cbd0bb1 in term 0.
I20260808 02:13:22.601284 27439 raft_consensus.cc:2454] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 471fb1b38509487facacea4e2cbd0bb1 in term 0.
I20260808 02:13:22.601694 27515 leader_election.cc:302] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 471fb1b38509487facacea4e2cbd0bb1, fa52b3cc791a439b88acb9dc9a3b7766; no voters:
I20260808 02:13:22.602034 27820 raft_consensus.cc:2781] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:22.602100 27820 raft_consensus.cc:482] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:22.602128 27820 raft_consensus.cc:3037] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:22.603179 27820 raft_consensus.cc:504] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.603600 27820 leader_election.cc:288] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [CANDIDATE]: Term 1 election: Requested vote from peers fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129:33317), ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895)
I20260808 02:13:22.603968 27439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c8ba77acdf4ff59e8217ce66b26a7a" candidate_uuid: "471fb1b38509487facacea4e2cbd0bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa52b3cc791a439b88acb9dc9a3b7766"
I20260808 02:13:22.604096 27439 raft_consensus.cc:3037] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:22.604079 27735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c8ba77acdf4ff59e8217ce66b26a7a" candidate_uuid: "471fb1b38509487facacea4e2cbd0bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc2a42726d147e8b34e8dd90cadc66c"
I20260808 02:13:22.604216 27735 raft_consensus.cc:3037] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:22.605193 27439 raft_consensus.cc:2454] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 471fb1b38509487facacea4e2cbd0bb1 in term 1.
I20260808 02:13:22.605469 27735 raft_consensus.cc:2454] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 471fb1b38509487facacea4e2cbd0bb1 in term 1.
I20260808 02:13:22.605489 27515 leader_election.cc:302] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 471fb1b38509487facacea4e2cbd0bb1, fa52b3cc791a439b88acb9dc9a3b7766; no voters:
I20260808 02:13:22.605784 27820 raft_consensus.cc:2781] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:22.606034 27820 raft_consensus.cc:686] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [term 1 LEADER]: Becoming Leader. State: Replica: 471fb1b38509487facacea4e2cbd0bb1, State: Running, Role: LEADER
I20260808 02:13:22.606328 27820 consensus_queue.cc:237] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.608188 27863 tablet_replica.cc:439] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: TxnStatusTablet state changed. Reason: New leader 471fb1b38509487facacea4e2cbd0bb1. Latest consensus state: current_term: 1 leader_uuid: "471fb1b38509487facacea4e2cbd0bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } } }
I20260808 02:13:22.608304 27863 tablet_replica.cc:442] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:13:22.608733 27865 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:13:22.608932 27865 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:13:22.609355 27269 catalog_manager.cc:5495] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130). New cstate: current_term: 1 leader_uuid: "471fb1b38509487facacea4e2cbd0bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:22.624147 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.619883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.18.184.130:37679)
0808 02:13:22.620027 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.620032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.620231 (+ 199us) server_negotiation.cc:407] Connection header received
0808 02:13:22.620470 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.620473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.620533 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.620675 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:22.620953 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.621601 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.622560 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.622813 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.622934 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.623290 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:22.623715 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:22.623881 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.623923 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.623982 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:13:22.624147 27855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.619744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:37679 (local address 127.0.0.1:53362)
0808 02:13:22.620143 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.620158 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:22.620303 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:22.620692 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:22.620696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:22.620703 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:13:22.620822 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.620829 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.621741 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.621748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.622425 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.622434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.622946 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.622950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.623064 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.623111 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:13:22.623825 (+ 714us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:13:22.623843 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 02:13:22.623911 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:13:22.624004 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224}
I20260808 02:13:22.627477 27735 raft_consensus.cc:1261] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Refusing update from remote peer 471fb1b38509487facacea4e2cbd0bb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:22.627568 27439 raft_consensus.cc:1261] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Refusing update from remote peer 471fb1b38509487facacea4e2cbd0bb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:22.628152 27863 consensus_queue.cc:1035] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:22.628386 27820 consensus_queue.cc:1035] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:22.630343 27818 tablet_replica.cc:439] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766: TxnStatusTablet state changed. Reason: New leader 471fb1b38509487facacea4e2cbd0bb1. Latest consensus state: current_term: 1 leader_uuid: "471fb1b38509487facacea4e2cbd0bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } } }
I20260808 02:13:22.630493 27818 tablet_replica.cc:442] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:22.632485 27821 tablet_replica.cc:439] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c: TxnStatusTablet state changed. Reason: New leader 471fb1b38509487facacea4e2cbd0bb1. Latest consensus state: current_term: 1 leader_uuid: "471fb1b38509487facacea4e2cbd0bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } } }
I20260808 02:13:22.632645 27821 tablet_replica.cc:442] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:22.634512 27863 tablet_replica.cc:439] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "471fb1b38509487facacea4e2cbd0bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } } }
I20260808 02:13:22.634670 27863 tablet_replica.cc:442] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:13:22.634908 27827 mvcc.cc:205] Tried to move back new op lower bound from 7316091709955862528 to 7316091709878112256. Current Snapshot: MvccSnapshot[applied={T|T < 7316091709955862528}]
I20260808 02:13:22.635514 27818 tablet_replica.cc:439] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "471fb1b38509487facacea4e2cbd0bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } } }
I20260808 02:13:22.635525 27870 tablet_replica.cc:439] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "471fb1b38509487facacea4e2cbd0bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } } }
I20260808 02:13:22.635641 27818 tablet_replica.cc:442] T 03c8ba77acdf4ff59e8217ce66b26a7a P fa52b3cc791a439b88acb9dc9a3b7766: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:22.635779 27870 tablet_replica.cc:442] T 03c8ba77acdf4ff59e8217ce66b26a7a P 471fb1b38509487facacea4e2cbd0bb1: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:13:22.637212 27829 mvcc.cc:205] Tried to move back new op lower bound from 7316091709955862528 to 7316091709878112256. Current Snapshot: MvccSnapshot[applied={T|T < 7316091709955862528}]
I20260808 02:13:22.637409 27821 tablet_replica.cc:439] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "471fb1b38509487facacea4e2cbd0bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } } }
I20260808 02:13:22.637542 27821 tablet_replica.cc:442] T 03c8ba77acdf4ff59e8217ce66b26a7a P ddc2a42726d147e8b34e8dd90cadc66c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:22.644168 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.639532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50724 (local address 127.18.184.131:38895)
0808 02:13:22.639670 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.639678 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.639803 (+ 125us) server_negotiation.cc:407] Connection header received
0808 02:13:22.639989 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.639992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.640070 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.640192 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:22.640507 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.641086 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.642335 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.642568 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.642782 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.643195 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:22.643673 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:22.643791 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.643908 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.643963 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 02:13:22.656090 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.652342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:56717 (local address 127.18.184.130:37679)
0808 02:13:22.652493 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.652498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.652534 (+ 36us) server_negotiation.cc:407] Connection header received
0808 02:13:22.652680 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.652683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.652737 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.652818 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:22.653059 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.653874 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.654849 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.655201 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.655306 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.655590 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:22.655676 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:22.655752 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.655894 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.655953 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:13:22.656373 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.652063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:37679 (local address 127.18.184.131:56717)
0808 02:13:22.652469 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.652483 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:22.652578 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:22.652837 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:22.652839 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:22.652845 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:13:22.652956 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.652960 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.654008 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.654011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.654671 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.654679 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.655316 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.655318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.655405 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.655452 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:13:22.655791 (+ 339us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:13:22.655798 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:13:22.655862 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 02:13:22.655934 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":116,"threads_started":1}
I20260808 02:13:22.665858 27439 raft_consensus.cc:1261] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Refusing update from remote peer ddc2a42726d147e8b34e8dd90cadc66c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:22.665872 27588 raft_consensus.cc:1261] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Refusing update from remote peer ddc2a42726d147e8b34e8dd90cadc66c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:22.666301 27821 consensus_queue.cc:1035] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [LEADER]: Connected to new peer: Peer: permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:22.666517 27847 consensus_queue.cc:1035] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:22.692955 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.687112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:38895 (local address 127.18.184.130:37683)
0808 02:13:22.688928 (+ 1816us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.688944 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:22.689056 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:22.689346 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:22.689349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:22.689357 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:13:22.689484 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.689490 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.690513 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.690516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.691500 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:22.691510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.692135 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.692139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:22.692226 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.692282 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:13:22.692613 (+ 331us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:13:22.692619 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:13:22.692687 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:13:22.692768 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1676,"thread_start_us":103,"threads_started":1}
I20260808 02:13:22.692945 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.687457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:37683 (local address 127.18.184.131:38895)
0808 02:13:22.687587 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.687592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.689029 (+ 1437us) server_negotiation.cc:407] Connection header received
0808 02:13:22.689175 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.689177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.689228 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.689302 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:22.689595 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.690389 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.691631 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.691995 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.692106 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.692431 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:22.692517 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:22.692593 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.692721 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.692780 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:13:22.725296 27269 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:22.725492 27269 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:22.728047 27269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727: Sending DeleteTablet for 3 replicas of tablet 9fa192794de04f1da636f53f0313cace
I20260808 02:13:22.728797 27418 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fa192794de04f1da636f53f0313cace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:22 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:57018
I20260808 02:13:22.729274 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fa192794de04f1da636f53f0313cace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:22 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:33136
I20260808 02:13:22.733955 27714 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fa192794de04f1da636f53f0313cace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:22 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:35570
I20260808 02:13:22.734375 27886 tablet_replica.cc:330] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c: stopping tablet replica
I20260808 02:13:22.734557 27886 raft_consensus.cc:2229] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:22.734946 27886 raft_consensus.cc:2258] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:22.735795 27886 ts_tablet_manager.cc:1905] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:22.739248 27886 ts_tablet_manager.cc:1918] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:22.739360 27886 log.cc:1196] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/9fa192794de04f1da636f53f0313cace
I20260808 02:13:22.739657 27886 ts_tablet_manager.cc:1939] T 9fa192794de04f1da636f53f0313cace P ddc2a42726d147e8b34e8dd90cadc66c: Deleting consensus metadata
I20260808 02:13:22.740490 27882 tablet_replica.cc:330] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766: stopping tablet replica
I20260808 02:13:22.740526 27258 catalog_manager.cc:4841] TS ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895): tablet 9fa192794de04f1da636f53f0313cace (table test-table [id=a5f545d16dd6461aaf8b4cbfdcadfc29]) successfully deleted
I20260808 02:13:22.740649 27882 raft_consensus.cc:2229] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:22.740777 27882 raft_consensus.cc:2258] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:22.741065 27883 tablet_replica.cc:330] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1: stopping tablet replica
I20260808 02:13:22.741225 27883 raft_consensus.cc:2229] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:22.741346 27883 raft_consensus.cc:2258] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:22.741750 27882 ts_tablet_manager.cc:1905] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:22.742074 27883 ts_tablet_manager.cc:1905] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:22.745010 27882 ts_tablet_manager.cc:1918] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:22.745179 27882 log.cc:1196] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/wal/wals/9fa192794de04f1da636f53f0313cace
I20260808 02:13:22.745133 27883 ts_tablet_manager.cc:1918] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:22.745254 27883 log.cc:1196] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/9fa192794de04f1da636f53f0313cace
I20260808 02:13:22.745546 27883 ts_tablet_manager.cc:1939] T 9fa192794de04f1da636f53f0313cace P 471fb1b38509487facacea4e2cbd0bb1: Deleting consensus metadata
I20260808 02:13:22.745863 27882 ts_tablet_manager.cc:1939] T 9fa192794de04f1da636f53f0313cace P fa52b3cc791a439b88acb9dc9a3b7766: Deleting consensus metadata
I20260808 02:13:22.746414 27255 catalog_manager.cc:4841] TS 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130:37679): tablet 9fa192794de04f1da636f53f0313cace (table test-table [id=a5f545d16dd6461aaf8b4cbfdcadfc29]) successfully deleted
I20260808 02:13:22.746622 27255 catalog_manager.cc:4841] TS fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129:33317): tablet 9fa192794de04f1da636f53f0313cace (table test-table [id=a5f545d16dd6461aaf8b4cbfdcadfc29]) successfully deleted
Aug 08 02:13:22 dist-test-slave-9gcw krb5kdc[27221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:13:22 dist-test-slave-9gcw krb5kdc[27221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:22 dist-test-slave-9gcw krb5kdc[27221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:13:22.776054 27854 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:22.767664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58880 (local address 127.18.184.190:43385)
0808 02:13:22.767816 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.767821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.768067 (+ 246us) server_negotiation.cc:407] Connection header received
0808 02:13:22.768198 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.768201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.768258 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.768364 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:22.769279 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.770172 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.771416 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.771791 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.771913 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.775905 (+ 3992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58880: BlockingRecv error: recv got EOF from 127.0.0.1:58880 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 02:13:22 dist-test-slave-9gcw krb5kdc[27221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM
I20260808 02:13:22.792471 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.781609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58896 (local address 127.18.184.190:43385)
0808 02:13:22.781741 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.781746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.782019 (+ 273us) server_negotiation.cc:407] Connection header received
0808 02:13:22.782149 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.782153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.782220 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.782325 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:22.783135 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.783968 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.785236 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.785569 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.785676 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.788641 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:22.788645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:22.788648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:22.788682 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:22.790563 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:22.791061 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:22.791064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:22.791066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:22.791168 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:22.791483 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:22.791486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:22.791488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:22.791733 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:22.791891 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.792135 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.792277 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:13:22.796046 27269 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:22.796520 27269 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:22.802013 27418 tablet_service.cc:1467] Processing CreateTablet for tablet 0567b44f4ede43ce99dc2c719487e584 (DEFAULT_TABLE table=test-table [id=faf6284ee7fa47ed9ced7de9336c357b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:22.802183 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 0567b44f4ede43ce99dc2c719487e584 (DEFAULT_TABLE table=test-table [id=faf6284ee7fa47ed9ced7de9336c357b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:22.802366 27418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0567b44f4ede43ce99dc2c719487e584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:22.802477 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0567b44f4ede43ce99dc2c719487e584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:22.805428 27714 tablet_service.cc:1467] Processing CreateTablet for tablet 0567b44f4ede43ce99dc2c719487e584 (DEFAULT_TABLE table=test-table [id=faf6284ee7fa47ed9ced7de9336c357b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:22.805732 27714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0567b44f4ede43ce99dc2c719487e584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:22.806427 27812 tablet_bootstrap.cc:492] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766: Bootstrap starting.
I20260808 02:13:22.807365 27812 tablet_bootstrap.cc:654] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:22.807953 27813 tablet_bootstrap.cc:492] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1: Bootstrap starting.
I20260808 02:13:22.808250 27815 tablet_bootstrap.cc:492] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c: Bootstrap starting.
I20260808 02:13:22.808864 27813 tablet_bootstrap.cc:654] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:22.809242 27812 tablet_bootstrap.cc:492] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766: No bootstrap required, opened a new log
I20260808 02:13:22.809255 27815 tablet_bootstrap.cc:654] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:22.809334 27812 ts_tablet_manager.cc:1397] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 02:13:22.809782 27812 raft_consensus.cc:348] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.809934 27812 raft_consensus.cc:374] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:22.809980 27812 raft_consensus.cc:729] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa52b3cc791a439b88acb9dc9a3b7766, State: Initialized, Role: FOLLOWER
I20260808 02:13:22.810107 27812 consensus_queue.cc:260] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.810456 27812 ts_tablet_manager.cc:1428] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:22.810909 27813 tablet_bootstrap.cc:492] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1: No bootstrap required, opened a new log
I20260808 02:13:22.811028 27813 ts_tablet_manager.cc:1397] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 02:13:22.811498 27813 raft_consensus.cc:348] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.811616 27813 raft_consensus.cc:374] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:22.811641 27813 raft_consensus.cc:729] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 471fb1b38509487facacea4e2cbd0bb1, State: Initialized, Role: FOLLOWER
I20260808 02:13:22.811787 27813 consensus_queue.cc:260] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.812079 27813 ts_tablet_manager.cc:1428] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:22.813764 27815 tablet_bootstrap.cc:492] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c: No bootstrap required, opened a new log
I20260808 02:13:22.813879 27815 ts_tablet_manager.cc:1397] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260808 02:13:22.814402 27815 raft_consensus.cc:348] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.814529 27815 raft_consensus.cc:374] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:22.814577 27815 raft_consensus.cc:729] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddc2a42726d147e8b34e8dd90cadc66c, State: Initialized, Role: FOLLOWER
I20260808 02:13:22.814713 27815 consensus_queue.cc:260] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.815022 27815 ts_tablet_manager.cc:1428] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:13:22.846135 27870 raft_consensus.cc:482] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:22.846274 27870 raft_consensus.cc:504] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.846783 27870 leader_election.cc:288] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129:33317), ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895)
I20260808 02:13:22.847254 27735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0567b44f4ede43ce99dc2c719487e584" candidate_uuid: "471fb1b38509487facacea4e2cbd0bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" is_pre_election: true
I20260808 02:13:22.847223 27439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0567b44f4ede43ce99dc2c719487e584" candidate_uuid: "471fb1b38509487facacea4e2cbd0bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" is_pre_election: true
I20260808 02:13:22.847402 27439 raft_consensus.cc:2454] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 471fb1b38509487facacea4e2cbd0bb1 in term 0.
I20260808 02:13:22.847431 27735 raft_consensus.cc:2454] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 471fb1b38509487facacea4e2cbd0bb1 in term 0.
I20260808 02:13:22.847752 27518 leader_election.cc:302] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 471fb1b38509487facacea4e2cbd0bb1, ddc2a42726d147e8b34e8dd90cadc66c; no voters:
I20260808 02:13:22.847931 27870 raft_consensus.cc:2781] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:22.848006 27870 raft_consensus.cc:482] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:22.848073 27870 raft_consensus.cc:3037] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:22.849133 27870 raft_consensus.cc:504] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.849558 27870 leader_election.cc:288] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [CANDIDATE]: Term 1 election: Requested vote from peers fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129:33317), ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895)
I20260808 02:13:22.849924 27439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0567b44f4ede43ce99dc2c719487e584" candidate_uuid: "471fb1b38509487facacea4e2cbd0bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa52b3cc791a439b88acb9dc9a3b7766"
I20260808 02:13:22.849924 27735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0567b44f4ede43ce99dc2c719487e584" candidate_uuid: "471fb1b38509487facacea4e2cbd0bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc2a42726d147e8b34e8dd90cadc66c"
I20260808 02:13:22.850049 27439 raft_consensus.cc:3037] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:22.850049 27735 raft_consensus.cc:3037] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:22.851215 27735 raft_consensus.cc:2454] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 471fb1b38509487facacea4e2cbd0bb1 in term 1.
I20260808 02:13:22.851215 27439 raft_consensus.cc:2454] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 471fb1b38509487facacea4e2cbd0bb1 in term 1.
I20260808 02:13:22.851526 27518 leader_election.cc:302] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 471fb1b38509487facacea4e2cbd0bb1, ddc2a42726d147e8b34e8dd90cadc66c; no voters:
I20260808 02:13:22.851670 27870 raft_consensus.cc:2781] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:22.851755 27870 raft_consensus.cc:686] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 1 LEADER]: Becoming Leader. State: Replica: 471fb1b38509487facacea4e2cbd0bb1, State: Running, Role: LEADER
I20260808 02:13:22.851925 27870 consensus_queue.cc:237] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.854039 27269 catalog_manager.cc:5495] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130). New cstate: current_term: 1 leader_uuid: "471fb1b38509487facacea4e2cbd0bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:22.877079 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.873615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53364 (local address 127.18.184.130:37679)
0808 02:13:22.873749 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:22.873753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:22.873767 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:13:22.873822 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:22.873824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:22.873909 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:22.873990 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:22.874272 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.874873 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.875834 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:22.876076 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:22.876195 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:22.876503 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:22.876604 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:22.876678 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:22.876851 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 02:13:22.876924 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:13:22.879732 27735 raft_consensus.cc:1261] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Refusing update from remote peer 471fb1b38509487facacea4e2cbd0bb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:22.879833 27439 raft_consensus.cc:1261] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Refusing update from remote peer 471fb1b38509487facacea4e2cbd0bb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:22.880354 27870 consensus_queue.cc:1035] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:22.880529 27871 consensus_queue.cc:1035] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:22.891336 27271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:22.892445 27271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:22.894744 27271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727: Sending DeleteTablet for 3 replicas of tablet 0567b44f4ede43ce99dc2c719487e584
I20260808 02:13:22.895447 27714 tablet_service.cc:1514] Processing DeleteTablet for tablet 0567b44f4ede43ce99dc2c719487e584 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:22 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:35570
I20260808 02:13:22.895627 27886 tablet_replica.cc:330] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c: stopping tablet replica
I20260808 02:13:22.895571 27418 tablet_service.cc:1514] Processing DeleteTablet for tablet 0567b44f4ede43ce99dc2c719487e584 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:22 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:57018
I20260808 02:13:22.895699 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet 0567b44f4ede43ce99dc2c719487e584 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:22 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:33136
I20260808 02:13:22.895741 27886 raft_consensus.cc:2229] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:22.895789 27882 tablet_replica.cc:330] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766: stopping tablet replica
I20260808 02:13:22.895866 27886 raft_consensus.cc:2258] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:22.895900 27883 tablet_replica.cc:330] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1: stopping tablet replica
I20260808 02:13:22.895915 27882 raft_consensus.cc:2229] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:22.896024 27883 raft_consensus.cc:2229] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:22.896046 27882 raft_consensus.cc:2258] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:22.896354 27883 raft_consensus.cc:2258] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:22.896888 27882 ts_tablet_manager.cc:1905] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:22.897083 27883 ts_tablet_manager.cc:1905] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:22.899667 27882 ts_tablet_manager.cc:1918] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:22.899684 27883 ts_tablet_manager.cc:1918] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:22.899753 27883 log.cc:1196] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/0567b44f4ede43ce99dc2c719487e584
I20260808 02:13:22.899890 27882 log.cc:1196] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-0/wal/wals/0567b44f4ede43ce99dc2c719487e584
I20260808 02:13:22.900019 27883 ts_tablet_manager.cc:1939] T 0567b44f4ede43ce99dc2c719487e584 P 471fb1b38509487facacea4e2cbd0bb1: Deleting consensus metadata
I20260808 02:13:22.900303 27882 ts_tablet_manager.cc:1939] T 0567b44f4ede43ce99dc2c719487e584 P fa52b3cc791a439b88acb9dc9a3b7766: Deleting consensus metadata
I20260808 02:13:22.900838 27886 ts_tablet_manager.cc:1905] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:22.901609 27255 catalog_manager.cc:4841] TS 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130:37679): tablet 0567b44f4ede43ce99dc2c719487e584 (table test-table [id=faf6284ee7fa47ed9ced7de9336c357b]) successfully deleted
I20260808 02:13:22.903275 27886 ts_tablet_manager.cc:1918] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:22.903360 27886 log.cc:1196] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/0567b44f4ede43ce99dc2c719487e584
I20260808 02:13:22.903601 27886 ts_tablet_manager.cc:1939] T 0567b44f4ede43ce99dc2c719487e584 P ddc2a42726d147e8b34e8dd90cadc66c: Deleting consensus metadata
I20260808 02:13:22.904247 27258 catalog_manager.cc:4841] TS ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895): tablet 0567b44f4ede43ce99dc2c719487e584 (table test-table [id=faf6284ee7fa47ed9ced7de9336c357b]) successfully deleted
I20260808 02:13:22.904366 27255 catalog_manager.cc:4841] TS fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129:33317): tablet 0567b44f4ede43ce99dc2c719487e584 (table test-table [id=faf6284ee7fa47ed9ced7de9336c357b]) successfully deleted
I20260808 02:13:22.904695 27271 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:22.905184 27271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:22.910539 27418 tablet_service.cc:1467] Processing CreateTablet for tablet 5fea37da2b7e4098b8848dfbf14b6740 (DEFAULT_TABLE table=test-table [id=fca7961a83ea40d380f43483cf5feca9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:22.910856 27418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fea37da2b7e4098b8848dfbf14b6740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:22.913311 27812 tablet_bootstrap.cc:492] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766: Bootstrap starting.
I20260808 02:13:22.913686 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 5fea37da2b7e4098b8848dfbf14b6740 (DEFAULT_TABLE table=test-table [id=fca7961a83ea40d380f43483cf5feca9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:22.913960 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fea37da2b7e4098b8848dfbf14b6740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:22.914191 27812 tablet_bootstrap.cc:654] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:22.916338 27813 tablet_bootstrap.cc:492] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1: Bootstrap starting.
I20260808 02:13:22.916803 27714 tablet_service.cc:1467] Processing CreateTablet for tablet 5fea37da2b7e4098b8848dfbf14b6740 (DEFAULT_TABLE table=test-table [id=fca7961a83ea40d380f43483cf5feca9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:22.917111 27714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fea37da2b7e4098b8848dfbf14b6740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:22.917217 27813 tablet_bootstrap.cc:654] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:22.917711 27812 tablet_bootstrap.cc:492] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766: No bootstrap required, opened a new log
I20260808 02:13:22.917798 27812 ts_tablet_manager.cc:1397] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 02:13:22.918195 27812 raft_consensus.cc:348] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.918303 27812 raft_consensus.cc:374] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:22.918329 27812 raft_consensus.cc:729] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa52b3cc791a439b88acb9dc9a3b7766, State: Initialized, Role: FOLLOWER
I20260808 02:13:22.918437 27812 consensus_queue.cc:260] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.918668 27812 ts_tablet_manager.cc:1428] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:22.919986 27815 tablet_bootstrap.cc:492] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c: Bootstrap starting.
I20260808 02:13:22.920109 27813 tablet_bootstrap.cc:492] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1: No bootstrap required, opened a new log
I20260808 02:13:22.920197 27813 ts_tablet_manager.cc:1397] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 02:13:22.920578 27813 raft_consensus.cc:348] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.920711 27813 raft_consensus.cc:374] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:22.920765 27813 raft_consensus.cc:729] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 471fb1b38509487facacea4e2cbd0bb1, State: Initialized, Role: FOLLOWER
I20260808 02:13:22.920785 27815 tablet_bootstrap.cc:654] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:22.920907 27813 consensus_queue.cc:260] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.921241 27813 ts_tablet_manager.cc:1428] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:22.922086 27815 tablet_bootstrap.cc:492] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c: No bootstrap required, opened a new log
I20260808 02:13:22.922178 27815 ts_tablet_manager.cc:1397] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 02:13:22.922506 27815 raft_consensus.cc:348] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.922605 27815 raft_consensus.cc:374] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:22.922634 27815 raft_consensus.cc:729] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddc2a42726d147e8b34e8dd90cadc66c, State: Initialized, Role: FOLLOWER
I20260808 02:13:22.922791 27815 consensus_queue.cc:260] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:22.923108 27815 ts_tablet_manager.cc:1428] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:23.069854 27818 raft_consensus.cc:482] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:23.070022 27818 raft_consensus.cc:504] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:23.070427 27818 leader_election.cc:288] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130:37679), ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895)
I20260808 02:13:23.071002 27735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fea37da2b7e4098b8848dfbf14b6740" candidate_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" is_pre_election: true
I20260808 02:13:23.071026 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fea37da2b7e4098b8848dfbf14b6740" candidate_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471fb1b38509487facacea4e2cbd0bb1" is_pre_election: true
I20260808 02:13:23.071192 27588 raft_consensus.cc:2454] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa52b3cc791a439b88acb9dc9a3b7766 in term 0.
I20260808 02:13:23.071195 27735 raft_consensus.cc:2454] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa52b3cc791a439b88acb9dc9a3b7766 in term 0.
I20260808 02:13:23.071534 27369 leader_election.cc:302] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ddc2a42726d147e8b34e8dd90cadc66c, fa52b3cc791a439b88acb9dc9a3b7766; no voters:
I20260808 02:13:23.071727 27818 raft_consensus.cc:2781] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:23.071789 27818 raft_consensus.cc:482] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:23.071838 27818 raft_consensus.cc:3037] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:23.072880 27818 raft_consensus.cc:504] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:23.073299 27818 leader_election.cc:288] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [CANDIDATE]: Term 1 election: Requested vote from peers 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130:37679), ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895)
I20260808 02:13:23.073688 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fea37da2b7e4098b8848dfbf14b6740" candidate_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471fb1b38509487facacea4e2cbd0bb1"
I20260808 02:13:23.073705 27735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fea37da2b7e4098b8848dfbf14b6740" candidate_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc2a42726d147e8b34e8dd90cadc66c"
I20260808 02:13:23.073835 27735 raft_consensus.cc:3037] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:23.073833 27588 raft_consensus.cc:3037] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:23.074914 27735 raft_consensus.cc:2454] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa52b3cc791a439b88acb9dc9a3b7766 in term 1.
I20260808 02:13:23.075022 27588 raft_consensus.cc:2454] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa52b3cc791a439b88acb9dc9a3b7766 in term 1.
I20260808 02:13:23.075243 27369 leader_election.cc:302] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ddc2a42726d147e8b34e8dd90cadc66c, fa52b3cc791a439b88acb9dc9a3b7766; no voters:
I20260808 02:13:23.075443 27818 raft_consensus.cc:2781] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:23.075908 27818 raft_consensus.cc:686] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 LEADER]: Becoming Leader. State: Replica: fa52b3cc791a439b88acb9dc9a3b7766, State: Running, Role: LEADER
I20260808 02:13:23.076140 27818 consensus_queue.cc:237] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } }
I20260808 02:13:23.078156 27271 catalog_manager.cc:5495] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129). New cstate: current_term: 1 leader_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa52b3cc791a439b88acb9dc9a3b7766" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:23.127106 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.123500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52140 (local address 127.18.184.129:33317)
0808 02:13:23.123769 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:23.123781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:23.123827 (+ 46us) server_negotiation.cc:407] Connection header received
0808 02:13:23.123867 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:23.123870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:23.123912 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:23.124014 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:23.124278 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.124913 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.125858 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.126099 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.126214 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.126504 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:23.126586 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:23.126685 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:23.126802 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 02:13:23.126860 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260808 02:13:23.137822 27915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.134319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:37679 (local address 127.18.184.129:41765)
0808 02:13:23.134700 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.134713 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:23.134815 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:23.135047 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:23.135049 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:23.135055 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:13:23.135148 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.135152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.135957 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.135960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.136694 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.136705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.137115 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.137117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.137230 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.137278 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:13:23.137569 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:13:23.137574 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:13:23.137624 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 02:13:23.137695 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":145,"threads_started":1}
I20260808 02:13:23.137810 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.134479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:41765 (local address 127.18.184.130:37679)
0808 02:13:23.134619 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:23.134623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:23.134779 (+ 156us) server_negotiation.cc:407] Connection header received
0808 02:13:23.134909 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:23.134911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:23.134957 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:23.135023 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:23.135235 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.135852 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.136800 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.137017 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.137094 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.137425 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:23.137493 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:23.137549 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:23.137653 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 02:13:23.137698 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:13:23.144162 27588 raft_consensus.cc:1261] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Refusing update from remote peer fa52b3cc791a439b88acb9dc9a3b7766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:23.144163 27735 raft_consensus.cc:1261] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Refusing update from remote peer fa52b3cc791a439b88acb9dc9a3b7766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:23.144587 27818 consensus_queue.cc:1035] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "471fb1b38509487facacea4e2cbd0bb1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 37679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:23.144747 27909 consensus_queue.cc:1035] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddc2a42726d147e8b34e8dd90cadc66c" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 38895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:23.162685 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.159281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:33317 (local address 127.18.184.130:57249)
0808 02:13:23.159402 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.159418 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:23.159493 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:23.159877 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:23.159880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:23.159888 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:13:23.159992 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.159998 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.160747 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.160751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.161412 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.161421 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.161926 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.161929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.162018 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.162055 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:13:23.162407 (+ 352us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:13:23.162415 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:13:23.162479 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 02:13:23.162563 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 02:13:23.162685 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.159354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:57249 (local address 127.18.184.129:33317)
0808 02:13:23.159658 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:23.159665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:23.159683 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:13:23.159728 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:23.159731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:23.159788 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:23.159863 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:23.160108 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.160629 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.161535 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.161823 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.161928 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.162216 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:23.162298 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:23.162401 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:23.162488 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 02:13:23.162543 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
I20260808 02:13:23.183689 27271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:23.183874 27271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:23.186281 27271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4710f8ac53bc40b4930d98c032d04727: Sending DeleteTablet for 3 replicas of tablet 5fea37da2b7e4098b8848dfbf14b6740
I20260808 02:13:23.186996 27418 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fea37da2b7e4098b8848dfbf14b6740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:23 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:57018
I20260808 02:13:23.187023 27714 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fea37da2b7e4098b8848dfbf14b6740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:23 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:35570
I20260808 02:13:23.187191 27882 tablet_replica.cc:330] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766: stopping tablet replica
I20260808 02:13:23.187160 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fea37da2b7e4098b8848dfbf14b6740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:23 UTC) from {username='oryx', principal='oryx/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:33136
I20260808 02:13:23.187220 27886 tablet_replica.cc:330] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c: stopping tablet replica
I20260808 02:13:23.187332 27882 raft_consensus.cc:2229] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:23.187379 27883 tablet_replica.cc:330] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1: stopping tablet replica
I20260808 02:13:23.187366 27886 raft_consensus.cc:2229] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:23.187495 27886 raft_consensus.cc:2258] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:23.187485 27883 raft_consensus.cc:2229] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:23.187506 27882 raft_consensus.cc:2258] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:23.187598 27883 raft_consensus.cc:2258] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:23.188295 27886 ts_tablet_manager.cc:1905] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:23.188295 27883 ts_tablet_manager.cc:1905] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:23.188679 27882 ts_tablet_manager.cc:1905] T 5fea37da2b7e4098b8848dfbf14b6740 P fa52b3cc791a439b88acb9dc9a3b7766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:23.188783 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 27349
I20260808 02:13:23.190914 27883 ts_tablet_manager.cc:1918] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:23.190914 27886 ts_tablet_manager.cc:1918] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:23.190989 27886 log.cc:1196] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/5fea37da2b7e4098b8848dfbf14b6740
I20260808 02:13:23.190989 27883 log.cc:1196] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/5fea37da2b7e4098b8848dfbf14b6740
I20260808 02:13:23.191259 27883 ts_tablet_manager.cc:1939] T 5fea37da2b7e4098b8848dfbf14b6740 P 471fb1b38509487facacea4e2cbd0bb1: Deleting consensus metadata
I20260808 02:13:23.191258 27886 ts_tablet_manager.cc:1939] T 5fea37da2b7e4098b8848dfbf14b6740 P ddc2a42726d147e8b34e8dd90cadc66c: Deleting consensus metadata
I20260808 02:13:23.192272 27258 catalog_manager.cc:4841] TS ddc2a42726d147e8b34e8dd90cadc66c (127.18.184.131:38895): tablet 5fea37da2b7e4098b8848dfbf14b6740 (table test-table [id=fca7961a83ea40d380f43483cf5feca9]) successfully deleted
I20260808 02:13:23.193948 27255 catalog_manager.cc:4841] TS 471fb1b38509487facacea4e2cbd0bb1 (127.18.184.130:37679): tablet 5fea37da2b7e4098b8848dfbf14b6740 (table test-table [id=fca7961a83ea40d380f43483cf5feca9]) successfully deleted
W20260808 02:13:23.200686 27255 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:33317 (error 108)
I20260808 02:13:23.200975 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 27499
I20260808 02:13:23.203194 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.202695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:33317 (local address 127.0.0.1:52150)
0808 02:13:23.202936 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.203035 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
W20260808 02:13:23.204010 27255 catalog_manager.cc:4584] TS fa52b3cc791a439b88acb9dc9a3b7766 (127.18.184.129:33317): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5fea37da2b7e4098b8848dfbf14b6740: Network error: Client connection negotiation failed: client connection to 127.18.184.129:33317: connect: Connection refused (error 111)
I20260808 02:13:23.211150 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 27645
I20260808 02:13:23.220753 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 27235
2026-08-08T02:13:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3869 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:23 dist-test-slave-9gcw krb5kdc[27925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:23 dist-test-slave-9gcw krb5kdc[27925](info): set up 2 sockets
Aug 08 02:13:23 dist-test-slave-9gcw krb5kdc[27925](info): commencing operation
krb5kdc: starting...
W20260808 02:13:24.361919 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:13:24 dist-test-slave-9gcw krb5kdc[27925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:13:24.505278 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:37083
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:37083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:24.623411 27938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:24.623786 27938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:24.623869 27938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:24.628221 27938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:24.628333 27938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:24.628368 27938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:24.628410 27938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:24.628445 27938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:24.634218 27938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:37083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:37083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:24.635488 27938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:24.636606 27938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:24.646027 27943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:24.646044 27944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:24.646031 27946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:24.646481 27938 server_base.cc:1034] running on GCE node
I20260808 02:13:24.647135 27938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:24.648329 27938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:24.649500 27938 hybrid_clock.cc:648] HybridClock initialized: now 1786155204649478 us; error 27 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T02:13:24Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1440 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:24 dist-test-slave-9gcw krb5kdc[27951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:24 dist-test-slave-9gcw krb5kdc[27951](info): set up 2 sockets
Aug 08 02:13:24 dist-test-slave-9gcw krb5kdc[27951](info): commencing operation
krb5kdc: starting...
W20260808 02:13:25.773437 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:13:25 dist-test-slave-9gcw krb5kdc[27951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:13:25.916806 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:39267
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:39267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:26.032895 27964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:26.033210 27964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:26.033263 27964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:26.037302 27964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:26.037375 27964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:26.037395 27964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:26.037412 27964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:26.037428 27964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:26.042891 27964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:39267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:39267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:26.044096 27964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:26.045094 27964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:26.054211 27970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:26.054175 27971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:26.054376 27964 server_base.cc:1034] running on GCE node
W20260808 02:13:26.054570 27973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:26.055153 27964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:26.056358 27964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:26.057518 27964 hybrid_clock.cc:648] HybridClock initialized: now 1786155206057486 us; error 47 us; skew 500 ppm
Aug 08 02:13:26 dist-test-slave-9gcw krb5kdc[27951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:26.060583 27964 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:13:26.061645 27964 webserver.cc:460] Webserver started at http://127.18.184.190:36053/ using document root <none> and password file <none>
I20260808 02:13:26.062224 27964 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:26.062327 27964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:26.062572 27964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:26.064740 27964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "d910d6d2986f4a5aa02d03b32d3d9230"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "7dde11f652f0bf8a5cf29d14bc4ff9f2"
server_key_iv: "750a474795ef4fcc3a759c3bfccbb35e"
server_key_version: "encryptionkey@0"
I20260808 02:13:26.065392 27964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "d910d6d2986f4a5aa02d03b32d3d9230"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "7dde11f652f0bf8a5cf29d14bc4ff9f2"
server_key_iv: "750a474795ef4fcc3a759c3bfccbb35e"
server_key_version: "encryptionkey@0"
I20260808 02:13:26.069604 27964 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:13:26.072167 27980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:26.073632 27964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:26.073848 27964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "d910d6d2986f4a5aa02d03b32d3d9230"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "7dde11f652f0bf8a5cf29d14bc4ff9f2"
server_key_iv: "750a474795ef4fcc3a759c3bfccbb35e"
server_key_version: "encryptionkey@0"
I20260808 02:13:26.073971 27964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:26.093662 27964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:26.097049 27964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:26.097309 27964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:26.104822 27964 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:39267
I20260808 02:13:26.104852 28036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:39267 every 8 connection(s)
I20260808 02:13:26.105919 27964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:26.109205 28037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:26.114780 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 27964
I20260808 02:13:26.114952 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:26.115283 19170 external_mini_cluster.cc:1428] Setting key 57f43bdc78da95a076d8b73e9665d3d8
I20260808 02:13:26.115999 28037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230: Bootstrap starting.
I20260808 02:13:26.118835 28037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:26.119741 28037 log.cc:824] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:26.122079 28037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230: No bootstrap required, opened a new log
Aug 08 02:13:26 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:26.126580 28037 raft_consensus.cc:348] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39267 } }
I20260808 02:13:26.126896 28037 raft_consensus.cc:374] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:26.126991 28037 raft_consensus.cc:729] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d910d6d2986f4a5aa02d03b32d3d9230, State: Initialized, Role: FOLLOWER
I20260808 02:13:26.127590 28037 consensus_queue.cc:260] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39267 } }
I20260808 02:13:26.127789 28037 raft_consensus.cc:388] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:26.127880 28037 raft_consensus.cc:482] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:26.128015 28037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:26.129108 28037 raft_consensus.cc:504] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39267 } }
I20260808 02:13:26.129591 28037 leader_election.cc:302] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d910d6d2986f4a5aa02d03b32d3d9230; no voters:
I20260808 02:13:26.130034 28037 leader_election.cc:288] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:26.130162 28042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:26.130470 28042 raft_consensus.cc:686] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 1 LEADER]: Becoming Leader. State: Replica: d910d6d2986f4a5aa02d03b32d3d9230, State: Running, Role: LEADER
I20260808 02:13:26.130837 28042 consensus_queue.cc:237] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39267 } }
I20260808 02:13:26.131065 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:26.116961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.18.184.190:39267)
0808 02:13:26.117457 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:26.117473 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:26.117510 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:13:26.118238 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:26.118257 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:26.118618 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:26.119009 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:26.119899 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.120932 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.121904 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.122439 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.122617 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:26.125434 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:26.125439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:26.125454 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:26.125486 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:26.127482 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:26.128068 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:26.128074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:26.128078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:26.128256 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:26.128563 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:26.128566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:26.128568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:26.128995 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:26.129236 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:26.129582 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 02:13:26.129977 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":167,"threads_started":1}
I20260808 02:13:26.131491 28037 sys_catalog.cc:564] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:26.132781 28043 sys_catalog.cc:455] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39267 } } }
I20260808 02:13:26.132776 28044 sys_catalog.cc:455] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d910d6d2986f4a5aa02d03b32d3d9230. Latest consensus state: current_term: 1 leader_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39267 } } }
I20260808 02:13:26.132930 28044 sys_catalog.cc:458] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:26.132930 28043 sys_catalog.cc:458] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:26.133642 28052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:26.137372 28052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:26.143520 28052 catalog_manager.cc:1347] Generated new cluster ID: c2dbc46fd0af4d8bbd7f3b68d48a5fb6
I20260808 02:13:26.143600 28052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:26.151190 28052 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:26.152158 28052 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:26.162535 28052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230: Generated new TSK 0
I20260808 02:13:26.163280 28052 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:13:26.209522 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:39267
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:26.347576 28065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:26.347900 28065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:26.347980 28065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:26.352067 28065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:26.352172 28065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:26.352267 28065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:26.357721 28065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:39267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:26.358850 28065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:26.359928 28065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:26.370188 28071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:26.370157 28075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:26.370148 28073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:26.370543 28065 server_base.cc:1034] running on GCE node
I20260808 02:13:26.371013 28065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:26.371660 28065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:26.372859 28065 hybrid_clock.cc:648] HybridClock initialized: now 1786155206372845 us; error 27 us; skew 500 ppm
Aug 08 02:13:26 dist-test-slave-9gcw krb5kdc[27951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:26.375813 28065 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:13:26.376749 28065 webserver.cc:460] Webserver started at http://127.18.184.129:40735/ using document root <none> and password file <none>
I20260808 02:13:26.377468 28065 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:26.377568 28065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:26.377804 28065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:26.379930 28065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "a1c7af78034c4a558d37731cf286c51f"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "f3127bed6189999a46622d2319147860"
server_key_iv: "c08165fd443188b66c03d8f679f198f5"
server_key_version: "encryptionkey@0"
I20260808 02:13:26.380517 28065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "a1c7af78034c4a558d37731cf286c51f"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "f3127bed6189999a46622d2319147860"
server_key_iv: "c08165fd443188b66c03d8f679f198f5"
server_key_version: "encryptionkey@0"
I20260808 02:13:26.384346 28065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:13:26.387068 28082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:26.388427 28065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:26.388603 28065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "a1c7af78034c4a558d37731cf286c51f"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "f3127bed6189999a46622d2319147860"
server_key_iv: "c08165fd443188b66c03d8f679f198f5"
server_key_version: "encryptionkey@0"
I20260808 02:13:26.388800 28065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:26.415844 28065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:26.419338 28065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:26.419564 28065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:26.420256 28065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:26.421233 28065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:26.421301 28065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:26.421373 28065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:26.421416 28065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:26.431059 28065 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:37005
I20260808 02:13:26.431221 28206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:37005 every 8 connection(s)
I20260808 02:13:26.432124 28065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:26.439288 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 28065
I20260808 02:13:26.439402 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:26.439750 19170 external_mini_cluster.cc:1428] Setting key d93851c74ba3b3b06c480709333e524a
Aug 08 02:13:26 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:26.447101 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:26.434369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:59331 (local address 127.18.184.190:39267)
0808 02:13:26.434560 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:26.434566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:26.435776 (+ 1210us) server_negotiation.cc:407] Connection header received
0808 02:13:26.436674 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:26.436677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:26.436737 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:26.436825 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:26.438148 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.438697 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.439954 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.440201 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.440319 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:26.443326 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:26.443329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:26.443332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:26.443364 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:26.444924 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:26.445535 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:26.445539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:26.445541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:26.445627 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:26.445990 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:26.445994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:26.445995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:26.446183 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:26.446290 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:26.446793 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 02:13:26.446925 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 02:13:26.447891 28209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:26.434834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:59331)
0808 02:13:26.435586 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:26.435630 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:26.436453 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:26.437014 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:26.437021 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:26.437493 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:26.438001 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:26.438016 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.438820 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.438823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:26.439786 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:26.439821 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.440367 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.440371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:26.440588 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:26.441355 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:26.441379 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:26.443155 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:26.445066 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:26.445072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:26.445087 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:26.445422 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:26.445734 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:26.445737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:26.445739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:26.445885 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:26.446290 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:26.446297 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:26.446551 (+ 254us) client_negotiation.cc:769] Sending connection context
0808 02:13:26.446735 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 02:13:26.447055 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":132,"threads_started":1}
I20260808 02:13:26.449204 28207 heartbeater.cc:347] Connected to a master server at 127.18.184.190:39267
I20260808 02:13:26.449478 28207 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:26.450146 28207 heartbeater.cc:511] Master 127.18.184.190:39267 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:13:26.451809 27994 ts_manager.cc:194] Registered new tserver with Master: a1c7af78034c4a558d37731cf286c51f (127.18.184.129:37005)
I20260808 02:13:26.453203 27994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:59331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:13:26.473533 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:39267
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:26.597559 28213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:26.598011 28213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:26.598063 28213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:26.602308 28213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:26.602394 28213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:26.602460 28213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:26.608165 28213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:39267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:26.609421 28213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:26.610488 28213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:26.621260 28221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:26.621260 28219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:26.621512 28213 server_base.cc:1034] running on GCE node
W20260808 02:13:26.621344 28218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:26.622215 28213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:26.622923 28213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:26.624115 28213 hybrid_clock.cc:648] HybridClock initialized: now 1786155206624089 us; error 36 us; skew 500 ppm
Aug 08 02:13:26 dist-test-slave-9gcw krb5kdc[27951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:26.627394 28213 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:13:26.628512 28213 webserver.cc:460] Webserver started at http://127.18.184.130:45775/ using document root <none> and password file <none>
I20260808 02:13:26.629096 28213 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:26.629222 28213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:26.629419 28213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:26.631645 28213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "e67bafe1e5dc4f409a7669292c777f0f"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "8db36cb943b0bbe61bea31c52ce11e27"
server_key_iv: "79b0eeae5e8845420d387ecd1fd10b48"
server_key_version: "encryptionkey@0"
I20260808 02:13:26.632302 28213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "e67bafe1e5dc4f409a7669292c777f0f"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "8db36cb943b0bbe61bea31c52ce11e27"
server_key_iv: "79b0eeae5e8845420d387ecd1fd10b48"
server_key_version: "encryptionkey@0"
I20260808 02:13:26.636574 28213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:13:26.639526 28228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:26.641004 28213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:26.641120 28213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "e67bafe1e5dc4f409a7669292c777f0f"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "8db36cb943b0bbe61bea31c52ce11e27"
server_key_iv: "79b0eeae5e8845420d387ecd1fd10b48"
server_key_version: "encryptionkey@0"
I20260808 02:13:26.641307 28213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:26.680837 28213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:26.684536 28213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:26.684803 28213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:26.685562 28213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:26.686554 28213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:26.686623 28213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:26.686710 28213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:26.686757 28213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:26.698135 28213 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:39643
I20260808 02:13:26.698248 28349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:39643 every 8 connection(s)
I20260808 02:13:26.699187 28213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:26.702826 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 28213
I20260808 02:13:26.702953 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:26.703287 19170 external_mini_cluster.cc:1428] Setting key a7994693699a91cc31c01bef06cb340d
Aug 08 02:13:26 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:13:26.715704 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:26.701700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:38547 (local address 127.18.184.190:39267)
0808 02:13:26.701859 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:26.701863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:26.703182 (+ 1319us) server_negotiation.cc:407] Connection header received
0808 02:13:26.704239 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:26.704243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:26.704299 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:26.704383 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:26.705878 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.706455 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.707938 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.708178 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.708295 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:26.711421 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:26.711424 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:26.711426 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:26.711454 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:26.712967 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:26.713719 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:26.713722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:26.713723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:26.713812 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:26.714255 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:26.714258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:26.714259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:26.714451 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:26.714532 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:26.715288 (+ 756us) server_negotiation.cc:299] Negotiation successful
0808 02:13:26.715489 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:13:26.716550 28352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:26.702140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:38547)
0808 02:13:26.702986 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:26.703045 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:26.704001 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:26.704560 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:26.704567 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:26.704987 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:26.705711 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:26.705735 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.706598 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.706604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:26.707804 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:26.707815 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.708349 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.708354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:26.708527 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:26.709209 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:26.709238 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:26.711212 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:26.713098 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:26.713106 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:26.713124 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:26.713542 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:26.713925 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:26.713930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:26.713932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:26.714129 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:26.714570 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:26.714577 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:26.714962 (+ 385us) client_negotiation.cc:769] Sending connection context
0808 02:13:26.715172 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 02:13:26.715583 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":139,"threads_started":1}
I20260808 02:13:26.718150 28350 heartbeater.cc:347] Connected to a master server at 127.18.184.190:39267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
I20260808 02:13:26.718475 28350 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:26.719055 28350 heartbeater.cc:511] Master 127.18.184.190:39267 requested a full tablet report, sending...
I20260808 02:13:26.720359 27994 ts_manager.cc:194] Registered new tserver with Master: e67bafe1e5dc4f409a7669292c777f0f (127.18.184.130:39643)
I20260808 02:13:26.721290 27994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:38547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:13:26.738699 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:39267
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:26.859299 28355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:26.859594 28355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:26.859678 28355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:26.863999 28355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:26.864069 28355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:26.864166 28355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:26.869742 28355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:39267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:26.870942 28355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:26.872061 28355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:26.882403 28361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:26.882421 28363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:26.882592 28360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:26.883272 28355 server_base.cc:1034] running on GCE node
I20260808 02:13:26.883810 28355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:26.884526 28355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:26.885773 28355 hybrid_clock.cc:648] HybridClock initialized: now 1786155206885740 us; error 55 us; skew 500 ppm
Aug 08 02:13:26 dist-test-slave-9gcw krb5kdc[27951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:26.888895 28355 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:13:26.889955 28355 webserver.cc:460] Webserver started at http://127.18.184.131:40863/ using document root <none> and password file <none>
I20260808 02:13:26.890587 28355 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:26.890694 28355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:26.890946 28355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:26.893224 28355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "7f7d22c1f87043dbae04337208d3c035"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "077ca44f6cfb8129c5c817e461f8fc14"
server_key_iv: "40e4cba1d4c198865c52f8753ad3f537"
server_key_version: "encryptionkey@0"
I20260808 02:13:26.893842 28355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "7f7d22c1f87043dbae04337208d3c035"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "077ca44f6cfb8129c5c817e461f8fc14"
server_key_iv: "40e4cba1d4c198865c52f8753ad3f537"
server_key_version: "encryptionkey@0"
I20260808 02:13:26.897936 28355 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 02:13:26.900517 28370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:26.901929 28355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:13:26.902081 28355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "7f7d22c1f87043dbae04337208d3c035"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "077ca44f6cfb8129c5c817e461f8fc14"
server_key_iv: "40e4cba1d4c198865c52f8753ad3f537"
server_key_version: "encryptionkey@0"
I20260808 02:13:26.902208 28355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:26.917986 28355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:26.921590 28355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:26.921820 28355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:26.922503 28355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:26.923763 28355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:26.923857 28355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:26.923920 28355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:26.923961 28355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:26.935096 28355 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:40105
I20260808 02:13:26.935173 28491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:40105 every 8 connection(s)
I20260808 02:13:26.936075 28355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:26.936734 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 28355
I20260808 02:13:26.936889 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:26.937245 19170 external_mini_cluster.cc:1428] Setting key 2d568e6546d1ab03efe23dce4bd2d63e
Aug 08 02:13:26 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:26.950217 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:26.938177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:37197 (local address 127.18.184.190:39267)
0808 02:13:26.938328 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:26.938334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:26.939482 (+ 1148us) server_negotiation.cc:407] Connection header received
0808 02:13:26.940500 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:26.940503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:26.940558 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:26.940637 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:26.941953 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.942502 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.943676 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.943918 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.944032 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:26.946719 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:26.946723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:26.946726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:26.946759 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:26.948048 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:26.948581 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:26.948584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:26.948586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:26.948669 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:26.949017 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:26.949021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:26.949023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:26.949305 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:26.949414 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:26.949928 (+ 514us) server_negotiation.cc:299] Negotiation successful
0808 02:13:26.950052 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:13:26.951359 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:26.938627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:37197)
0808 02:13:26.939333 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:26.939376 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:26.940296 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:26.940823 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:26.940831 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:26.941288 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:26.941811 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:26.941825 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.942635 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.942639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:26.943547 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:26.943556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:26.944079 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:26.944083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:26.944263 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:26.944945 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:26.944961 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:26.946551 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:26.948172 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:26.948178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:26.948193 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:26.948462 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:26.948768 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:26.948771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:26.948773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:26.948918 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:26.949412 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:26.949419 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:26.949667 (+ 248us) client_negotiation.cc:769] Sending connection context
0808 02:13:26.949872 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 02:13:26.950383 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":160,"threads_started":1}
I20260808 02:13:26.952525 28492 heartbeater.cc:347] Connected to a master server at 127.18.184.190:39267
I20260808 02:13:26.952788 28492 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:26.953346 28492 heartbeater.cc:511] Master 127.18.184.190:39267 requested a full tablet report, sending...
I20260808 02:13:26.954594 27994 ts_manager.cc:194] Registered new tserver with Master: 7f7d22c1f87043dbae04337208d3c035 (127.18.184.131:40105)
I20260808 02:13:26.955287 27994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:37197
I20260808 02:13:26.964422 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab.
Entry for principal oryx/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab.
WARNING: no policy specified for oryx/127.18.184.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab.
Entry for principal oryx/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab.
WARNING: no policy specified for oryx/127.18.184.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.184.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab.
Entry for principal oryx/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab.
I20260808 02:13:27.056761 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 28065
I20260808 02:13:27.064847 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 28213
I20260808 02:13:27.072755 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 28355
I20260808 02:13:27.080327 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 27964
I20260808 02:13:27.089489 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:39267
--webserver_interface=127.18.184.190
--webserver_port=36053
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:39267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:27.206149 28501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:27.206494 28501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:27.206578 28501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:27.210616 28501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:27.210717 28501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:27.210774 28501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:27.210834 28501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:27.210873 28501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:27.216321 28501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:39267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:39267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=36053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:27.217463 28501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:27.218504 28501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:27.227910 28506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:27.227929 28509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:27.227990 28501 server_base.cc:1034] running on GCE node
W20260808 02:13:27.227922 28507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:27.228791 28501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:27.230105 28501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:27.231328 28501 hybrid_clock.cc:648] HybridClock initialized: now 1786155207231310 us; error 29 us; skew 500 ppm
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155207, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:27.234448 28501 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:13:27.235482 28501 webserver.cc:460] Webserver started at http://127.18.184.190:36053/ using document root <none> and password file <none>
I20260808 02:13:27.236093 28501 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:27.236212 28501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:27.240674 28501 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:27.242915 28516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:27.244290 28501 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:27.244400 28501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "d910d6d2986f4a5aa02d03b32d3d9230"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "7dde11f652f0bf8a5cf29d14bc4ff9f2"
server_key_iv: "750a474795ef4fcc3a759c3bfccbb35e"
server_key_version: "encryptionkey@0"
I20260808 02:13:27.244940 28501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:27.262709 28501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:27.266367 28501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:27.266539 28501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:27.274246 28501 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:39267
I20260808 02:13:27.274322 28573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:39267 every 8 connection(s)
I20260808 02:13:27.275413 28501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:27.277230 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 28501
I20260808 02:13:27.277702 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:37005
--local_ip_for_outbound_sockets=127.18.184.129
--tserver_master_addrs=127.18.184.190:39267
--webserver_port=40735
--webserver_interface=127.18.184.129
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.184.129
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:13:27.280303 28574 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:13:27.282975 28574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230: Bootstrap starting.
I20260808 02:13:27.307307 28574 log.cc:824] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:27.315850 28574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:13:27.316323 28574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230: Bootstrap complete.
I20260808 02:13:27.322285 28574 raft_consensus.cc:348] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39267 } }
I20260808 02:13:27.322846 28574 raft_consensus.cc:729] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d910d6d2986f4a5aa02d03b32d3d9230, State: Initialized, Role: FOLLOWER
I20260808 02:13:27.323428 28574 consensus_queue.cc:260] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39267 } }
I20260808 02:13:27.323616 28574 raft_consensus.cc:388] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:27.323702 28574 raft_consensus.cc:482] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:27.323812 28574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:13:27.326629 28574 raft_consensus.cc:504] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39267 } }
I20260808 02:13:27.327030 28574 leader_election.cc:302] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d910d6d2986f4a5aa02d03b32d3d9230; no voters:
I20260808 02:13:27.327436 28574 leader_election.cc:288] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:13:27.327598 28578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:13:27.327919 28578 raft_consensus.cc:686] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [term 2 LEADER]: Becoming Leader. State: Replica: d910d6d2986f4a5aa02d03b32d3d9230, State: Running, Role: LEADER
I20260808 02:13:27.328243 28578 consensus_queue.cc:237] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39267 } }
I20260808 02:13:27.328833 28574 sys_catalog.cc:564] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:27.329280 28581 sys_catalog.cc:455] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d910d6d2986f4a5aa02d03b32d3d9230. Latest consensus state: current_term: 2 leader_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39267 } } }
I20260808 02:13:27.329412 28581 sys_catalog.cc:458] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:27.329703 28579 sys_catalog.cc:455] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d6d2986f4a5aa02d03b32d3d9230" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39267 } } }
I20260808 02:13:27.329790 28579 sys_catalog.cc:458] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:27.330217 28583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:27.333658 28583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:27.334545 28583 catalog_manager.cc:1259] Loaded cluster ID: c2dbc46fd0af4d8bbd7f3b68d48a5fb6
I20260808 02:13:27.334611 28583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:27.337024 28583 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:27.337744 28583 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d910d6d2986f4a5aa02d03b32d3d9230: Loaded TSK: 0
I20260808 02:13:27.338969 28583 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:13:27.413977 28576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:27.414283 28576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:27.414362 28576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:27.418503 28576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:27.418605 28576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:27.418704 28576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:27.424222 28576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.129.keytab
--principal=oryx/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:37005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=40735
--tserver_master_addrs=127.18.184.190:39267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:27.425436 28576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:27.426462 28576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:27.437805 28601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:27.437847 28602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:27.437867 28604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:27.438417 28576 server_base.cc:1034] running on GCE node
I20260808 02:13:27.438886 28576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:27.439620 28576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:27.440840 28576 hybrid_clock.cc:648] HybridClock initialized: now 1786155207440811 us; error 48 us; skew 500 ppm
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155207, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:27.444154 28576 init.cc:377] Logged in from keytab as oryx/127.18.184.129@KRBTEST.COM (short username oryx)
I20260808 02:13:27.445297 28576 webserver.cc:460] Webserver started at http://127.18.184.129:40735/ using document root <none> and password file <none>
I20260808 02:13:27.445984 28576 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:27.446075 28576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:27.450614 28576 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 02:13:27.452972 28611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:27.454406 28576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:13:27.454552 28576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "a1c7af78034c4a558d37731cf286c51f"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "f3127bed6189999a46622d2319147860"
server_key_iv: "c08165fd443188b66c03d8f679f198f5"
server_key_version: "encryptionkey@0"
I20260808 02:13:27.455145 28576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:27.473850 28576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:27.477783 28576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:27.478039 28576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:27.478747 28576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:27.479687 28576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:27.479740 28576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:27.479830 28576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:27.479866 28576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:27.490291 28576 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:37005
I20260808 02:13:27.490336 28739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:37005 every 8 connection(s)
I20260808 02:13:27.491439 28576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:27.496711 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 28576
I20260808 02:13:27.497354 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:39643
--local_ip_for_outbound_sockets=127.18.184.130
--tserver_master_addrs=127.18.184.190:39267
--webserver_port=45775
--webserver_interface=127.18.184.130
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.184.130
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:27.506709 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:27.493942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:57833)
0808 02:13:27.494670 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:27.494726 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:27.495571 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:27.496746 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:27.496752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:27.497308 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:27.497955 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.497972 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.499772 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.499779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.501129 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.501191 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.502338 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.502341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.502481 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.503286 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:27.503303 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:27.505904 (+ 2601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":147,"threads_started":1}
W20260808 02:13:27.507272 28742 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:27.493814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:57833 (local address 127.18.184.190:39267)
0808 02:13:27.494284 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:27.494301 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:27.494859 (+ 558us) server_negotiation.cc:407] Connection header received
0808 02:13:27.495961 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:27.495973 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:27.496267 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:27.496641 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:27.498159 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.499605 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.501328 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.502154 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.502384 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.506252 (+ 3868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:57833: BlockingRecv error: recv got EOF from 127.18.184.129:57833 (error 108)
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":131,"threads_started":1}
W20260808 02:13:27.507396 28740 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:39267 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.184.190:39267: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:27.513693 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:27.507988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:36899)
0808 02:13:27.508117 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:27.508133 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:27.508211 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:27.508489 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:27.508491 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:27.508627 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:27.508726 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.508730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.509910 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.509912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.510493 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.510501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.511083 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.511086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.511157 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.511654 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:27.511665 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:27.513550 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:13:27.513849 28742 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:27.508083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:36899 (local address 127.18.184.190:39267)
0808 02:13:27.508228 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:27.508233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:27.508251 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:13:27.508302 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:27.508307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:27.508368 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:27.508474 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:27.508877 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.509798 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.510611 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.510972 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.511077 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.513687 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:36899: BlockingRecv error: recv got EOF from 127.18.184.129:36899 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:27.519934 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:27.514424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:49259)
0808 02:13:27.514590 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:27.514604 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:27.514681 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:27.514932 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:27.514935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:27.515106 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:27.515213 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.515219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.515930 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.515933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.516848 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.516859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.517330 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.517334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.517420 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.517876 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:27.517891 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:27.519794 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101}
W20260808 02:13:27.520041 28742 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:27.514520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:49259 (local address 127.18.184.190:39267)
0808 02:13:27.514632 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:27.514635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:27.514647 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:13:27.514766 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:27.514768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:27.514806 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:27.514891 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:27.515323 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.515828 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.516969 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.517229 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.517303 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.519927 (+ 2624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:49259: BlockingRecv error: recv got EOF from 127.18.184.129:49259 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 02:13:27.520255 28740 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:13:27.620393 28744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:27.620744 28744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:27.620824 28744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:27.625154 28744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:27.625252 28744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:27.625350 28744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:27.631222 28744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.130.keytab
--principal=oryx/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:39643
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=45775
--tserver_master_addrs=127.18.184.190:39267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:27.632436 28744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:27.633564 28744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:27.643754 28750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:27.643754 28752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:27.643821 28749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:27.644218 28744 server_base.cc:1034] running on GCE node
I20260808 02:13:27.644618 28744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:27.645370 28744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:27.646574 28744 hybrid_clock.cc:648] HybridClock initialized: now 1786155207646555 us; error 30 us; skew 500 ppm
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155207, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:27.649770 28744 init.cc:377] Logged in from keytab as oryx/127.18.184.130@KRBTEST.COM (short username oryx)
I20260808 02:13:27.650717 28744 webserver.cc:460] Webserver started at http://127.18.184.130:45775/ using document root <none> and password file <none>
I20260808 02:13:27.651307 28744 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:27.651407 28744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:27.655390 28744 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 02:13:27.657598 28759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:27.658926 28744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:27.659060 28744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "e67bafe1e5dc4f409a7669292c777f0f"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "8db36cb943b0bbe61bea31c52ce11e27"
server_key_iv: "79b0eeae5e8845420d387ecd1fd10b48"
server_key_version: "encryptionkey@0"
I20260808 02:13:27.659623 28744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:27.686785 28744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:27.690341 28744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:27.690563 28744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:27.691205 28744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:27.692076 28744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:27.692150 28744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:27.692224 28744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:27.692281 28744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:27.701673 28744 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:39643
I20260808 02:13:27.701717 28874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:39643 every 8 connection(s)
I20260808 02:13:27.702790 28744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:27.706729 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 28744
I20260808 02:13:27.707337 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:40105
--local_ip_for_outbound_sockets=127.18.184.131
--tserver_master_addrs=127.18.184.190:39267
--webserver_port=40863
--webserver_interface=127.18.184.131
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.184.131
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:13:27.717607 28742 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:27.704853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:48801 (local address 127.18.184.190:39267)
0808 02:13:27.704998 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:27.705002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:27.706244 (+ 1242us) server_negotiation.cc:407] Connection header received
0808 02:13:27.707760 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:27.707768 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:27.707825 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:27.707922 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:27.709856 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.710746 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.712631 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.712880 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.712972 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.717470 (+ 4498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:48801: BlockingRecv error: recv got EOF from 127.18.184.130:48801 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:13:27.718344 28877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:27.705246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:48801)
0808 02:13:27.706040 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:27.706088 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:27.707429 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:27.708172 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:27.708186 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:27.708823 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:27.709670 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.709691 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.710912 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.710915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.712477 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.712492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.713080 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.713087 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.713366 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.714252 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:27.714281 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:27.717325 (+ 3044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":151,"threads_started":1}
W20260808 02:13:27.718838 28875 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:39267 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.184.190:39267: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:27.724953 28877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:27.719399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:54091)
0808 02:13:27.719520 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:27.719537 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:27.719651 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:27.720010 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:27.720012 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:27.720194 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:27.720292 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.720298 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.721053 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.721058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.721697 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.721707 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.722170 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.722174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.722280 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.722802 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:27.722814 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:27.724799 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 02:13:27.725051 28742 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:27.719478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:54091 (local address 127.18.184.190:39267)
0808 02:13:27.719626 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:27.719630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:27.719645 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:13:27.719849 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:27.719854 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:27.719904 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:27.719992 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:27.720410 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.720939 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.721825 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.722046 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.722154 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.724930 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:54091: BlockingRecv error: recv got EOF from 127.18.184.130:54091 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:27.731235 28877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:27.725769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:48705)
0808 02:13:27.725956 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:27.725973 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:27.726052 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:27.726366 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:27.726369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:27.726537 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:27.726609 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.726614 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.727398 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.727403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.727935 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.727943 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.728401 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.728404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.728485 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.729015 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:27.729026 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:27.731078 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 02:13:27.731325 28742 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:27.726033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:48705 (local address 127.18.184.190:39267)
0808 02:13:27.726176 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:27.726179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:27.726190 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:13:27.726227 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:27.726229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:27.726264 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:27.726350 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:27.726729 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.727287 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.728060 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.728283 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.728389 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.731195 (+ 2806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:48705: BlockingRecv error: recv got EOF from 127.18.184.130:48705 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 02:13:27.731737 28875 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:13:27.832413 28878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:27.832772 28878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:27.832863 28878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:27.837356 28878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:27.837452 28878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:27.837550 28878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:27.844898 28878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/krb5kdc/oryx_127.18.184.131.keytab
--principal=oryx/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:40105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=40863
--tserver_master_addrs=127.18.184.190:39267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:27.846418 28878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:27.847725 28878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:27.859754 28884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:27.859789 28885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:27.859877 28878 server_base.cc:1034] running on GCE node
W20260808 02:13:27.859798 28887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:27.860523 28878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:27.861306 28878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:27.862490 28878 hybrid_clock.cc:648] HybridClock initialized: now 1786155207862415 us; error 88 us; skew 500 ppm
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155207, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:27.865876 28878 init.cc:377] Logged in from keytab as oryx/127.18.184.131@KRBTEST.COM (short username oryx)
I20260808 02:13:27.866865 28878 webserver.cc:460] Webserver started at http://127.18.184.131:40863/ using document root <none> and password file <none>
I20260808 02:13:27.867456 28878 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:27.867534 28878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:27.872139 28878 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:13:27.875176 28894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:27.876773 28878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:27.876915 28878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "7f7d22c1f87043dbae04337208d3c035"
format_stamp: "Formatted at 2026-08-08 02:13:26 on dist-test-slave-9gcw"
server_key: "077ca44f6cfb8129c5c817e461f8fc14"
server_key_iv: "40e4cba1d4c198865c52f8753ad3f537"
server_key_version: "encryptionkey@0"
I20260808 02:13:27.877579 28878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:27.904806 28878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:27.908532 28878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:27.908715 28878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:27.909461 28878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:27.910341 28878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:27.910391 28878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:27.910458 28878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:27.910492 28878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:27.919979 28878 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:40105
I20260808 02:13:27.920004 29025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:40105 every 8 connection(s)
I20260808 02:13:27.921049 28878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:27.927347 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 28878
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:13:27.934054 28742 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:27.923252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:38853 (local address 127.18.184.190:39267)
0808 02:13:27.923448 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:27.923452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:27.924624 (+ 1172us) server_negotiation.cc:407] Connection header received
0808 02:13:27.925497 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:27.925500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:27.925565 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:27.925655 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:27.926976 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.927592 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.929067 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.929577 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.929697 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.933916 (+ 4219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:38853: BlockingRecv error: recv got EOF from 127.18.184.131:38853 (error 108)
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":84}
I20260808 02:13:27.934919 29028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:27.923696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:38853)
0808 02:13:27.924443 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:27.924489 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:27.925285 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:27.925852 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:27.925859 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:27.926269 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:27.926795 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.926812 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.927735 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.927739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.928961 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.928971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.929747 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.929752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.929932 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.930669 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:27.930690 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:27.933724 (+ 3034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":143,"threads_started":1}
W20260808 02:13:27.935496 29026 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:39267 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.184.190:39267: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
I20260808 02:13:27.939098 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:27.928665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43074 (local address 127.18.184.190:39267)
0808 02:13:27.928985 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:27.928988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:27.929000 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:13:27.929057 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:27.929060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:27.929107 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:27.929285 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:27.930272 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.931080 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.932431 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.932652 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.932780 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.934513 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:27.934520 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:27.934541 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:27.934575 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:27.936697 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:27.937205 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:27.937211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:27.937217 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:27.937355 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:27.937669 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:27.937674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:27.937675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:27.938043 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:27.938265 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:27.938568 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 02:13:27.938884 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":253,"thread_start_us":113,"threads_started":1}
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:27.941450 29028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:27.935995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:40031)
0808 02:13:27.936093 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:27.936108 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:27.936196 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:27.936439 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:27.936442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:27.936609 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:27.936706 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.936711 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.937532 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.937536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.938096 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.938104 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.938574 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.938577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.938697 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.939264 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:27.939281 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:27.941344 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 02:13:27.941632 28742 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:27.936089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:40031 (local address 127.18.184.190:39267)
0808 02:13:27.936200 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:27.936204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:27.936219 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:13:27.936282 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:27.936285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:27.936325 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:27.936422 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:27.936823 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.937424 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.938233 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.938457 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.938567 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.941492 (+ 2925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:40031: BlockingRecv error: recv got EOF from 127.18.184.131:40031 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:27 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:27.947726 29028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:27.942437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:56397)
0808 02:13:27.942564 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:27.942582 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:27.942676 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:27.942918 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:27.942922 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:27.943061 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:27.943165 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.943171 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.943876 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.943879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.944486 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:27.944495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.944942 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.944945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:27.945010 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.945665 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:27.945678 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:27.947611 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 02:13:27.948009 28742 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:27.942517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:56397 (local address 127.18.184.190:39267)
0808 02:13:27.942669 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:27.942672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:27.942684 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:13:27.942775 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:27.942777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:27.942819 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:27.942880 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:27.943266 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.943748 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.944602 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:27.944830 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:27.944910 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:27.947882 (+ 2972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:56397: BlockingRecv error: recv got EOF from 127.18.184.131:56397 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 02:13:27.947981 29026 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 02:13:28 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:28 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:28.527032 29031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:28.521070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:46633)
0808 02:13:28.521440 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:28.521457 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:28.521575 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:28.521908 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:28.521911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:28.522112 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:28.522236 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:28.522282 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.523151 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.523153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:28.523746 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:28.523754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.524176 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.524178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:28.524263 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:28.524807 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:28.524820 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:28.526854 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":101,"threads_started":1}
W20260808 02:13:28.527096 29032 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:28.521217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:46633 (local address 127.18.184.190:39267)
0808 02:13:28.521465 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:28.521470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:28.521540 (+ 70us) server_negotiation.cc:407] Connection header received
0808 02:13:28.521694 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:28.521697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:28.521762 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:28.521849 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:28.522409 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.523052 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.523862 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.524079 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.524158 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:28.526976 (+ 2818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:46633: BlockingRecv error: recv got EOF from 127.18.184.129:46633 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":86,"threads_started":1}
Aug 08 02:13:28 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:28 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:28.739477 29033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:28.732450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:38759)
0808 02:13:28.732870 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:28.732896 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:28.733044 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:28.733416 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:28.733419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:28.733663 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:28.733804 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:28.733811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.734698 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.734703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:28.735874 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:28.735888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.736524 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.736527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:28.736649 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:28.737317 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:28.737331 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:28.739350 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":88,"threads_started":1}
W20260808 02:13:28.739723 29032 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:28.732601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:38759 (local address 127.18.184.190:39267)
0808 02:13:28.732825 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:28.732831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:28.732988 (+ 157us) server_negotiation.cc:407] Connection header received
0808 02:13:28.733224 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:28.733228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:28.733287 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:28.733390 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:28.733944 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.734556 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.736029 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.736364 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.736498 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:28.739538 (+ 3040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:38759: BlockingRecv error: recv got EOF from 127.18.184.130:38759 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 08 02:13:28 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:28 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:28.954828 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:28.948761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:55299)
0808 02:13:28.949188 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:28.949207 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:28.949311 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:28.949590 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:28.949591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:28.949775 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:28.949902 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:28.949907 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.950639 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.950641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:28.951232 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:28.951240 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.951650 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.951653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:28.951745 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:28.952240 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:28.952254 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:28.954646 (+ 2392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":87,"threads_started":1}
W20260808 02:13:28.954929 29032 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:28.948916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:55299 (local address 127.18.184.190:39267)
0808 02:13:28.949057 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:28.949061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:28.949277 (+ 216us) server_negotiation.cc:407] Connection header received
0808 02:13:28.949378 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:28.949380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:28.949464 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:28.949555 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:28.950008 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.950544 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.951333 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.951548 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.951626 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:28.954815 (+ 3189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:55299: BlockingRecv error: recv got EOF from 127.18.184.131:55299 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 02:13:29 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:29.534655 29035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.528036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:39949)
0808 02:13:29.528362 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.528375 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.528541 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.528849 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.528852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.529041 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.529187 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.529198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.530187 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.530190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.530851 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.530860 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.531377 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.531379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.531444 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.532022 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.532035 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.534469 (+ 2434us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":102,"threads_started":1}
W20260808 02:13:29.534731 29036 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:29.528187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:39949 (local address 127.18.184.190:39267)
0808 02:13:29.528425 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.528428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.528441 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:13:29.528642 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.528644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.528692 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.528771 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.529372 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.530067 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.530995 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.531262 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.531355 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.534604 (+ 3249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:39949: BlockingRecv error: recv got EOF from 127.18.184.129:39949 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":64,"threads_started":1}
Aug 08 02:13:29 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:29.750172 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.740338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:51037)
0808 02:13:29.740600 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.740618 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.740753 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.741051 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.741053 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.741276 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.741377 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.741382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.742155 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.742158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.742728 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.742736 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.743347 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.743350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.743419 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.747596 (+ 4177us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.747620 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.749966 (+ 2346us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
W20260808 02:13:29.750245 29036 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:29.740425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:51037 (local address 127.18.184.190:39267)
0808 02:13:29.740571 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.740575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.740730 (+ 155us) server_negotiation.cc:407] Connection header received
0808 02:13:29.740842 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.740849 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.740898 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.741002 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.741483 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.742006 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.742905 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.743206 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.743311 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.750126 (+ 6815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:51037: BlockingRecv error: recv got EOF from 127.18.184.130:51037 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 02:13:29 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:29.962291 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.955934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:33093)
0808 02:13:29.956274 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.956292 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.956411 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.956714 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.956717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.956949 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.957047 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.957053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.957851 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.957857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.958503 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.958512 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.959030 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.959034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.959124 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.959705 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.959718 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.962038 (+ 2320us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":110,"threads_started":1}
W20260808 02:13:29.962337 29036 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:29.956075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:33093 (local address 127.18.184.190:39267)
0808 02:13:29.956253 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.956258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.956373 (+ 115us) server_negotiation.cc:407] Connection header received
0808 02:13:29.956532 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.956536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.956591 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.956686 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.957203 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.957732 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.958625 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.958896 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.959012 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.962210 (+ 3198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:33093: BlockingRecv error: recv got EOF from 127.18.184.131:33093 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 02:13:30 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:30 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:30.541939 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:30.535679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:35667)
0808 02:13:30.536149 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:30.536167 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:30.536298 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:30.536654 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:30.536658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:30.536942 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:30.537094 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:30.537101 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.538013 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.538020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:30.538622 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:30.538630 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.539150 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.539153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:30.539233 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:30.539761 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:30.539772 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:30.541802 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":87,"threads_started":1}
W20260808 02:13:30.542085 29040 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:30.535807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:35667 (local address 127.18.184.190:39267)
0808 02:13:30.536067 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:30.536074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:30.536265 (+ 191us) server_negotiation.cc:407] Connection header received
0808 02:13:30.536410 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:30.536413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:30.536479 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:30.536572 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:30.537287 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.537823 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.538788 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.539012 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.539091 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:30.541943 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:35667: BlockingRecv error: recv got EOF from 127.18.184.129:35667 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":96,"threads_started":1}
Aug 08 02:13:30 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:30 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:30.757652 29041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:30.751347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:51153)
0808 02:13:30.751663 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:30.751701 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:30.751844 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:30.752162 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:30.752165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:30.752382 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:30.752549 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:30.752561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.753589 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.753593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:30.754236 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:30.754246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.754710 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.754713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:30.754804 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:30.755432 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:30.755455 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:30.757516 (+ 2061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":117,"threads_started":1}
W20260808 02:13:30.757874 29040 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:30.751519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:51153 (local address 127.18.184.190:39267)
0808 02:13:30.751706 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:30.751710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:30.751777 (+ 67us) server_negotiation.cc:407] Connection header received
0808 02:13:30.751953 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:30.751956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:30.752003 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:30.752093 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:30.752719 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.753437 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.754368 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.754601 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.754687 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:30.757677 (+ 2990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:51153: BlockingRecv error: recv got EOF from 127.18.184.130:51153 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 02:13:30 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:30 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:30.973556 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:30.963297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:35163)
0808 02:13:30.963683 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:30.963699 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:30.963824 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:30.967696 (+ 3872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:30.967700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:30.967940 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:30.968073 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:30.968081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.969297 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.969300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:30.969917 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:30.969926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.970408 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.970412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:30.970504 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:30.971096 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:30.971118 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:30.973363 (+ 2245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":108,"threads_started":1}
W20260808 02:13:30.973578 29040 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:30.967227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:35163 (local address 127.18.184.190:39267)
0808 02:13:30.967396 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:30.967400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:30.967416 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:13:30.967460 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:30.967462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:30.967510 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:30.967629 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:30.968205 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.969095 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.970042 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.970277 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.970398 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:30.973475 (+ 3077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:35163: BlockingRecv error: recv got EOF from 127.18.184.131:35163 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 02:13:31 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:31 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:31.548820 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:31.542846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:49981)
0808 02:13:31.543150 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:31.543165 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:31.543285 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:31.543546 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:31.543548 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:31.543734 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:31.543836 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:31.543841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.544646 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.544649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:31.545336 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:31.545346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.545808 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.545811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:31.545880 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:31.546534 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:31.546548 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:31.548641 (+ 2093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":116,"threads_started":1}
W20260808 02:13:31.548978 29044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:31.542903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:49981 (local address 127.18.184.190:39267)
0808 02:13:31.543161 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:31.543164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:31.543214 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:13:31.543375 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:31.543378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:31.543423 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:31.543505 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:31.543958 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.544513 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.545453 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.545677 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.545781 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:31.548837 (+ 3056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:49981: BlockingRecv error: recv got EOF from 127.18.184.129:49981 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
Aug 08 02:13:31 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:31 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:31.764899 29045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:31.758552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:51389)
0808 02:13:31.758887 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:31.758902 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:31.759015 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:31.759289 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:31.759291 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:31.759498 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:31.759606 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:31.759612 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.760557 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.760559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:31.761245 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:31.761256 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.761737 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.761741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:31.761809 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:31.762405 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:31.762419 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:31.764768 (+ 2349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":94,"threads_started":1}
W20260808 02:13:31.764931 29044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:31.758703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:51389 (local address 127.18.184.190:39267)
0808 02:13:31.758858 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:31.758861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:31.759006 (+ 145us) server_negotiation.cc:407] Connection header received
0808 02:13:31.759111 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:31.759113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:31.759162 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:31.759231 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:31.759762 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.760419 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.761390 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.761614 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.761702 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:31.764837 (+ 3135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:51389: BlockingRecv error: recv got EOF from 127.18.184.130:51389 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 02:13:31 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:31 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:31.980809 29046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:31.974565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:45681)
0808 02:13:31.974899 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:31.974913 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:31.975016 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:31.975302 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:31.975305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:31.975540 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:31.975649 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:31.975656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.976570 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.976573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:31.977384 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:31.977394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.977915 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.977919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:31.978010 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:31.978677 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:31.978692 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:31.980632 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":160,"threads_started":1}
W20260808 02:13:31.980891 29044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:31.974664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:45681 (local address 127.18.184.190:39267)
0808 02:13:31.974800 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:31.974811 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:31.974983 (+ 172us) server_negotiation.cc:407] Connection header received
0808 02:13:31.975094 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:31.975096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:31.975145 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:31.975228 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:31.975840 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.976446 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.977518 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.977785 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.977913 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:31.980774 (+ 2861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:45681: BlockingRecv error: recv got EOF from 127.18.184.131:45681 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 02:13:32 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:32 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:32.556519 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:32.550321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:56551)
0808 02:13:32.550665 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:32.550745 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:32.550863 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:32.551168 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:32.551171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:32.551376 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:32.551483 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:32.551488 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.552488 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.552491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:32.553330 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:32.553341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.553819 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.553823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:32.553908 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:32.554413 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:32.554425 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:32.556340 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":98,"threads_started":1}
W20260808 02:13:32.556594 29048 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:32.550376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:56551 (local address 127.18.184.190:39267)
0808 02:13:32.550723 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:32.550727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:32.550839 (+ 112us) server_negotiation.cc:407] Connection header received
0808 02:13:32.550991 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:32.550994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:32.551044 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:32.551131 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:32.551751 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.552341 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.553451 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.553697 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.553811 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:32.556469 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:56551: BlockingRecv error: recv got EOF from 127.18.184.129:56551 (error 108)
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":113,"threads_started":1}
Aug 08 02:13:32 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:32 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:32.772778 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:32.766009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:50653)
0808 02:13:32.766415 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:32.766434 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:32.766591 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:32.767062 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:32.767067 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:32.767306 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:32.767413 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:32.767419 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.768173 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.768175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:32.768833 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:32.768843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.769541 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.769545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:32.769645 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:32.770371 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:32.770389 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:32.772619 (+ 2230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":118,"threads_started":1}
W20260808 02:13:32.773038 29048 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:32.766205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:50653 (local address 127.18.184.190:39267)
0808 02:13:32.766408 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:32.766414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:32.766541 (+ 127us) server_negotiation.cc:407] Connection header received
0808 02:13:32.766730 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:32.766736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:32.766802 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:32.766916 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:32.767520 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.768051 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.768998 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.769371 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.769491 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:32.772868 (+ 3377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:50653: BlockingRecv error: recv got EOF from 127.18.184.130:50653 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 08 02:13:32 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:32 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:32.987267 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:32.981698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:43621)
0808 02:13:32.982014 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:32.982028 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:32.982120 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:32.982359 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:32.982361 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:32.982540 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:32.982633 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:32.982638 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.983411 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.983413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:32.983981 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:32.983989 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.984420 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.984423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:32.984486 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:32.985029 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:32.985041 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:32.987075 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":148,"threads_started":1}
W20260808 02:13:32.987556 29048 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:32.981817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:43621 (local address 127.18.184.190:39267)
0808 02:13:32.981956 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:32.981960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:32.982105 (+ 145us) server_negotiation.cc:407] Connection header received
0808 02:13:32.982199 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:32.982201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:32.982248 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:32.982320 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:32.982755 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.983311 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.984089 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.984316 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.984394 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:32.987266 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:43621: BlockingRecv error: recv got EOF from 127.18.184.131:43621 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 02:13:33 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:33 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:33.563835 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:33.557486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:58533)
0808 02:13:33.557819 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:33.557833 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:33.557927 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:33.558310 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:33.558312 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:33.558528 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:33.558628 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:33.558633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.559478 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.559481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:33.560281 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:33.560292 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.560873 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.560876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:33.560951 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:33.561641 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:33.561654 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:33.563654 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":111,"threads_started":1}
W20260808 02:13:33.564002 29052 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:33.557592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:58533 (local address 127.18.184.190:39267)
0808 02:13:33.557894 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:33.557898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:33.557912 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:13:33.558066 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:33.558069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:33.558150 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:33.558247 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:33.558737 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.559341 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.560429 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.560722 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.560828 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:33.563860 (+ 3032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:58533: BlockingRecv error: recv got EOF from 127.18.184.129:58533 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":152,"threads_started":1}
Aug 08 02:13:33 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:33 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:33.779930 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:33.773683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:43663)
0808 02:13:33.773991 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:33.774004 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:33.774096 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:33.774339 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:33.774341 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:33.774536 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:33.774632 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:33.774637 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.775470 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.775473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:33.776016 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:33.776023 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.776553 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.776558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:33.776723 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:33.777381 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:33.777396 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:33.779788 (+ 2392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":124,"threads_started":1}
W20260808 02:13:33.780138 29052 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:33.773919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:43663 (local address 127.18.184.190:39267)
0808 02:13:33.774077 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:33.774080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:33.774097 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:13:33.774171 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:33.774174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:33.774220 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:33.774306 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:33.774766 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.775366 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.776135 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.776391 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.776480 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:33.779996 (+ 3516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:43663: BlockingRecv error: recv got EOF from 127.18.184.130:43663 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 02:13:33 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:33 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:33.994138 29054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:33.988075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:56059)
0808 02:13:33.988368 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:33.988381 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:33.988479 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:33.988705 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:33.988707 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:33.988904 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:33.989005 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:33.989010 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.989879 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.989882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:33.990463 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:33.990471 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.990909 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.990912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:33.990973 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:33.991587 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:33.991602 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:33.993992 (+ 2390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":87,"threads_started":1}
W20260808 02:13:33.994310 29052 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:33.988148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:56059 (local address 127.18.184.190:39267)
0808 02:13:33.988306 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:33.988310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:33.988462 (+ 152us) server_negotiation.cc:407] Connection header received
0808 02:13:33.988547 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:33.988549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:33.988598 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:33.988756 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:33.989205 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.989767 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.990568 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.990806 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.990888 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:33.994172 (+ 3284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:56059: BlockingRecv error: recv got EOF from 127.18.184.131:56059 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 02:13:34 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:34 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:34.571506 29055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:34.564830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:54435)
0808 02:13:34.565211 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:34.565224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:34.565353 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:34.565636 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:34.565639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:34.565896 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:34.566005 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:34.566011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.567004 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.567007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:34.567583 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:34.567592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.568275 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.568279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:34.568350 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:34.569080 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:34.569098 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:34.571377 (+ 2279us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":107,"threads_started":1}
W20260808 02:13:34.571803 29056 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:34.564982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:54435 (local address 127.18.184.190:39267)
0808 02:13:34.565302 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:34.565306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:34.565323 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:13:34.565450 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:34.565452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:34.565501 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:34.565598 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:34.566176 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.566880 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.567787 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.568087 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.568256 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:34.571547 (+ 3291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:54435: BlockingRecv error: recv got EOF from 127.18.184.129:54435 (error 108)
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":101,"threads_started":1}
Aug 08 02:13:34 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:34 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:34.788713 29057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:34.780998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:50577)
0808 02:13:34.781532 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:34.781553 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:34.781692 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:34.782173 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:34.782178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:34.782483 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:34.782626 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:34.782637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.783435 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.783442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:34.784673 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:34.784688 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.785449 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.785456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:34.785553 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:34.786184 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:34.786207 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:34.788581 (+ 2374us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":142,"threads_started":1}
W20260808 02:13:34.788975 29056 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:34.781518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:50577 (local address 127.18.184.190:39267)
0808 02:13:34.781669 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:34.781675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:34.781766 (+ 91us) server_negotiation.cc:407] Connection header received
0808 02:13:34.781820 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:34.781823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:34.781907 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:34.782018 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:34.782763 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.783303 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.784821 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.785283 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.785410 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:34.788805 (+ 3395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:50577: BlockingRecv error: recv got EOF from 127.18.184.130:50577 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 02:13:34 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:34 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:35.000639 29060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:34.995050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:37249)
0808 02:13:34.995435 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:34.995449 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:34.995555 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:34.995790 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:34.995792 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:34.995989 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:34.996090 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:34.996095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.996845 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.996848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:34.997474 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:34.997483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.997949 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.997951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:34.998011 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:34.998572 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:34.998584 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:35.000517 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":150,"threads_started":1}
W20260808 02:13:35.000949 29056 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:34.995180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:37249 (local address 127.18.184.190:39267)
0808 02:13:34.995336 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:34.995341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:34.995536 (+ 195us) server_negotiation.cc:407] Connection header received
0808 02:13:34.995636 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:34.995638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:34.995688 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:34.995775 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:34.996190 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.996704 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.997586 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.997842 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.997931 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:35.000823 (+ 2892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:37249: BlockingRecv error: recv got EOF from 127.18.184.131:37249 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 02:13:35 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:35 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:35.578279 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:35.572219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:45709)
0808 02:13:35.572507 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:35.572521 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:35.572609 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:35.573034 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:35.573036 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:35.573259 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:35.573366 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:35.573371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.574167 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.574169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:35.574732 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:35.574749 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.575199 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.575202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:35.575277 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:35.575981 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:35.575993 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:35.578132 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":101,"threads_started":1}
W20260808 02:13:35.578415 29062 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:35.572509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:45709 (local address 127.18.184.190:39267)
0808 02:13:35.572789 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:35.572793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:35.572807 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:13:35.572844 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:35.572846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:35.572889 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:35.572971 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:35.573478 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.574060 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.574849 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.575093 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.575200 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:35.578275 (+ 3075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:45709: BlockingRecv error: recv got EOF from 127.18.184.129:45709 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":94,"threads_started":1}
Aug 08 02:13:35 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:35 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:35.795318 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:35.789675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:57667)
0808 02:13:35.789987 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:35.790000 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:35.790101 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:35.790340 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:35.790341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:35.790530 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:35.790626 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:35.790631 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.791516 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.791518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:35.792064 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:35.792072 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.792494 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.792497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:35.792558 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:35.793060 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:35.793072 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:35.795187 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":101,"threads_started":1}
W20260808 02:13:35.795468 29062 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:35.789733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:57667 (local address 127.18.184.190:39267)
0808 02:13:35.789896 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:35.789902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:35.790081 (+ 179us) server_negotiation.cc:407] Connection header received
0808 02:13:35.790184 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:35.790186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:35.790235 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:35.790293 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:35.790761 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.791421 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.792172 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.792389 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.792466 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:35.795324 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:57667: BlockingRecv error: recv got EOF from 127.18.184.130:57667 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 02:13:36 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:36 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:36.007997 29064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:36.001623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:37209)
0808 02:13:36.002074 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:36.002089 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:36.002267 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:36.002568 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:36.002571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:36.002803 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:36.002900 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:36.002906 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.003679 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.003682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:36.004253 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:36.004261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.004727 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.004731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:36.004820 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:36.005463 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:36.005476 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:36.007870 (+ 2394us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":101,"threads_started":1}
W20260808 02:13:36.008208 29062 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:36.001803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:37209 (local address 127.18.184.190:39267)
0808 02:13:36.002015 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:36.002021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:36.002208 (+ 187us) server_negotiation.cc:407] Connection header received
0808 02:13:36.002391 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:36.002394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:36.002450 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:36.002540 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:36.003026 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.003560 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.004379 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.004608 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.004718 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:36.008055 (+ 3337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:37209: BlockingRecv error: recv got EOF from 127.18.184.131:37209 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 08 02:13:36 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:36 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:36.585597 29065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:36.579180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:33145)
0808 02:13:36.579534 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:36.579550 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:36.579725 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:36.580098 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:36.580100 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:36.580374 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:36.580491 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:36.580498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.581477 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.581480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:36.582126 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:36.582134 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.582619 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.582621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:36.582686 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:36.583259 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:36.583271 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:36.585467 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":119,"threads_started":1}
W20260808 02:13:36.585871 29066 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:36.579296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:33145 (local address 127.18.184.190:39267)
0808 02:13:36.579590 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:36.579595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:36.579610 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:13:36.579867 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:36.579870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:36.579931 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:36.580053 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:36.580621 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.581369 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.582283 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.582511 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.582603 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:36.585625 (+ 3022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:33145: BlockingRecv error: recv got EOF from 127.18.184.129:33145 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":110,"threads_started":1}
Aug 08 02:13:36 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:36 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:36.802116 29067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:36.796315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:35985)
0808 02:13:36.796615 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:36.796629 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:36.796737 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:36.796987 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:36.796989 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:36.797209 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:36.797333 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:36.797339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.798240 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.798242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:36.798817 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:36.798826 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.799281 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.799283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:36.799344 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:36.799946 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:36.799959 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:36.801996 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":95,"threads_started":1}
W20260808 02:13:36.802299 29066 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:36.796427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:35985 (local address 127.18.184.190:39267)
0808 02:13:36.796613 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:36.796624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:36.796718 (+ 94us) server_negotiation.cc:407] Connection header received
0808 02:13:36.796823 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:36.796825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:36.796871 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:36.796938 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:36.797452 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.798096 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.798926 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.799142 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.799237 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:36.802139 (+ 2902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:35985: BlockingRecv error: recv got EOF from 127.18.184.130:35985 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 08 02:13:37 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:37 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:37.014892 29069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:37.009011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:55721)
0808 02:13:37.009400 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:37.009416 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:37.009555 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:37.009917 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:37.009922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:37.010141 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:37.010288 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:37.010300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.011244 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.011248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:37.011830 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:37.011839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.012250 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.012253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:37.012317 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:37.012821 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:37.012832 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:37.014772 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":173,"threads_started":1}
W20260808 02:13:37.015033 29066 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:37.009229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:55721 (local address 127.18.184.190:39267)
0808 02:13:37.009374 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:37.009379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:37.009526 (+ 147us) server_negotiation.cc:407] Connection header received
0808 02:13:37.009655 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:37.009658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:37.009768 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:37.009877 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:37.010420 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.011136 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.011938 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.012150 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.012230 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:37.014912 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:55721: BlockingRecv error: recv got EOF from 127.18.184.131:55721 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 02:13:37 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:37 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:37.592993 29070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:37.586626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:46987)
0808 02:13:37.586984 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:37.586998 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:37.587128 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:37.587525 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:37.587528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:37.587730 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:37.587925 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:37.587935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.588704 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.588706 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:37.589334 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:37.589343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.589852 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.589854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:37.589916 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:37.590450 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:37.590461 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:37.592851 (+ 2390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":86,"threads_started":1}
W20260808 02:13:37.593289 29071 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:37.586687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:46987 (local address 127.18.184.190:39267)
0808 02:13:37.587114 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:37.587118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:37.587150 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:13:37.587298 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:37.587301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:37.587364 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:37.587460 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:37.588057 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.588603 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.589453 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.589724 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.589838 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:37.593022 (+ 3184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:46987: BlockingRecv error: recv got EOF from 127.18.184.129:46987 (error 108)
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":80,"threads_started":1}
Aug 08 02:13:37 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:37 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:37.809274 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:37.803083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:48689)
0808 02:13:37.803401 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:37.803415 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:37.803552 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:37.803918 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:37.803921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:37.804236 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:37.804339 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:37.804346 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.805124 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.805129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:37.805744 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:37.805753 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.806247 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.806250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:37.806325 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:37.806927 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:37.806939 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:37.809031 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":108,"threads_started":1}
W20260808 02:13:37.809317 29071 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:37.803136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:48689 (local address 127.18.184.190:39267)
0808 02:13:37.803317 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:37.803323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:37.803497 (+ 174us) server_negotiation.cc:407] Connection header received
0808 02:13:37.803660 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:37.803663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:37.803720 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:37.803859 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:37.804450 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.805010 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.805852 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.806074 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.806178 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:37.809207 (+ 3029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:48689: BlockingRecv error: recv got EOF from 127.18.184.130:48689 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 02:13:38 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:38 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:38.022209 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:38.015950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:33561)
0808 02:13:38.016283 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:38.016297 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:38.016395 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:38.016645 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:38.016647 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:38.016849 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:38.016948 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:38.016953 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.017726 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.017729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:38.018311 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:38.018321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.019030 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.019033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:38.019135 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:38.019734 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:38.019752 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:38.022072 (+ 2320us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":116,"threads_started":1}
W20260808 02:13:38.022404 29071 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:38.016130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:33561 (local address 127.18.184.190:39267)
0808 02:13:38.016278 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:38.016281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:38.016371 (+ 90us) server_negotiation.cc:407] Connection header received
0808 02:13:38.016481 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:38.016483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:38.016527 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:38.016601 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:38.017063 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.017622 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.018451 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.018885 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.019002 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:38.022254 (+ 3252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:33561: BlockingRecv error: recv got EOF from 127.18.184.131:33561 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 02:13:38 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:38 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:38.600682 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:38.594029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:60291)
0808 02:13:38.594533 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:38.594554 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:38.594691 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:38.595100 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:38.595106 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:38.595375 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:38.595512 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:38.595520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.596376 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.596378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:38.596935 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:38.596942 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.597450 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.597453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:38.597519 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:38.598063 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:38.598075 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:38.600557 (+ 2482us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":99,"threads_started":1}
W20260808 02:13:38.601022 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:38.594144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:60291 (local address 127.18.184.190:39267)
0808 02:13:38.594483 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:38.594490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:38.594657 (+ 167us) server_negotiation.cc:407] Connection header received
0808 02:13:38.594876 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:38.594879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:38.594952 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:38.595055 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:38.595649 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.596266 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.597041 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.597345 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.597433 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:38.600870 (+ 3437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:60291: BlockingRecv error: recv got EOF from 127.18.184.129:60291 (error 108)
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":138,"threads_started":1}
Aug 08 02:13:38 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:38 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:38.815847 29076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:38.810144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:56599)
0808 02:13:38.810500 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:38.810513 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:38.810609 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:38.810827 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:38.810829 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:38.811026 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:38.811122 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:38.811127 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.812043 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.812046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:38.812634 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:38.812644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.813106 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.813108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:38.813203 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:38.813800 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:38.813812 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:38.815710 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":100,"threads_started":1}
W20260808 02:13:38.815997 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:38.810165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:56599 (local address 127.18.184.190:39267)
0808 02:13:38.810349 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:38.810355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:38.810589 (+ 234us) server_negotiation.cc:407] Connection header received
0808 02:13:38.810692 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:38.810694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:38.810750 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:38.810817 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:38.811250 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.811838 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.812762 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.812991 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.813074 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:38.815869 (+ 2795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:56599: BlockingRecv error: recv got EOF from 127.18.184.130:56599 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 08 02:13:39 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:39 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:39.030140 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:39.023246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:50127)
0808 02:13:39.023581 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:39.023675 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:39.023808 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:39.024126 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:39.024129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:39.024333 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:39.024441 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:39.024447 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.025379 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.025384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:39.026442 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:39.026457 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.026955 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.026958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:39.027028 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:39.027751 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:39.027766 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:39.029961 (+ 2195us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":136,"threads_started":1}
W20260808 02:13:39.030215 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:39.023397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:50127 (local address 127.18.184.190:39267)
0808 02:13:39.023547 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:39.023551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:39.023784 (+ 233us) server_negotiation.cc:407] Connection header received
0808 02:13:39.023903 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:39.023905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:39.023958 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:39.024040 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:39.024562 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.025227 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.026579 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.026840 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.026920 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:39.030118 (+ 3198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:50127: BlockingRecv error: recv got EOF from 127.18.184.131:50127 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 02:13:39 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:39 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:39.607932 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:39.601721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:34823)
0808 02:13:39.602055 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:39.602069 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:39.602176 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:39.602498 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:39.602501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:39.602690 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:39.602801 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:39.602806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.603754 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.603757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:39.604312 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:39.604320 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.604797 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.604799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:39.604858 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:39.605565 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:39.605577 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:39.607809 (+ 2232us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":116,"threads_started":1}
W20260808 02:13:39.608156 29079 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:39.601842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:34823 (local address 127.18.184.190:39267)
0808 02:13:39.602117 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:39.602121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:39.602138 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:13:39.602315 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:39.602319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:39.602372 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:39.602467 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:39.602959 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.603567 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.604445 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.604678 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.604778 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:39.607965 (+ 3187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:34823: BlockingRecv error: recv got EOF from 127.18.184.129:34823 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":66,"threads_started":1}
Aug 08 02:13:39 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:39 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:39.827220 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:39.816865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:53959)
0808 02:13:39.817199 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:39.817214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:39.817322 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:39.817593 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:39.817595 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:39.817871 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:39.817982 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:39.817989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.818880 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.818883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:39.819500 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:39.819507 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.820199 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.820202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:39.820273 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:39.824757 (+ 4484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:39.824771 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:39.827024 (+ 2253us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":100,"threads_started":1}
W20260808 02:13:39.827316 29079 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:39.816975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:53959 (local address 127.18.184.190:39267)
0808 02:13:39.817206 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:39.817211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:39.817292 (+ 81us) server_negotiation.cc:407] Connection header received
0808 02:13:39.817414 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:39.817417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:39.817468 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:39.817549 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:39.818113 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.818760 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.819620 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.820060 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.820195 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:39.827172 (+ 6977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:53959: BlockingRecv error: recv got EOF from 127.18.184.130:53959 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 08 02:13:40 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:40 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:40.037739 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:40.031019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:42925)
0808 02:13:40.031351 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:40.031364 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:40.031470 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:40.031712 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:40.031714 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:40.031940 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:40.032041 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:40.032047 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.032936 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.032941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:40.033781 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:40.033794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.034348 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.034352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:40.034429 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:40.035204 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:40.035221 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:40.037544 (+ 2323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":92,"threads_started":1}
W20260808 02:13:40.037811 29079 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:40.031097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:42925 (local address 127.18.184.190:39267)
0808 02:13:40.031336 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:40.031340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:40.031444 (+ 104us) server_negotiation.cc:407] Connection header received
0808 02:13:40.031552 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:40.031555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:40.031603 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:40.031673 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:40.032193 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.032732 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.033949 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.034182 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.034282 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:40.037708 (+ 3426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:42925: BlockingRecv error: recv got EOF from 127.18.184.131:42925 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
Aug 08 02:13:40 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:40 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:40.615212 29083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:40.608932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:36515)
0808 02:13:40.609411 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:40.609430 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:40.609550 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:40.609895 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:40.609898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:40.610157 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:40.610264 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:40.610271 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.611102 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.611109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:40.611759 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:40.611769 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.612299 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.612303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:40.612404 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:40.612990 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:40.613002 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:40.615030 (+ 2028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":100,"threads_started":1}
W20260808 02:13:40.615329 29084 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:40.609014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:36515 (local address 127.18.184.190:39267)
0808 02:13:40.609360 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:40.609366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:40.609530 (+ 164us) server_negotiation.cc:407] Connection header received
0808 02:13:40.609655 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:40.609662 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:40.609735 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:40.609865 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:40.610402 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.610971 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.611900 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.612165 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.612291 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:40.615187 (+ 2896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:36515: BlockingRecv error: recv got EOF from 127.18.184.129:36515 (error 108)
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":124,"threads_started":1}
Aug 08 02:13:40 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:40 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:40.834806 29085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:40.828359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:39481)
0808 02:13:40.828660 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:40.828685 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:40.828790 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:40.829092 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:40.829094 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:40.829383 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:40.829486 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:40.829493 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.830452 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.830457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:40.831375 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:40.831386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.831985 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.831988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:40.832078 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:40.832597 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:40.832610 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:40.834625 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":113,"threads_started":1}
W20260808 02:13:40.834870 29084 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:40.828597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:39481 (local address 127.18.184.190:39267)
0808 02:13:40.828756 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:40.828761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:40.828782 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:13:40.828875 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:40.828877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:40.828921 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:40.829002 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:40.829627 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.830283 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.831496 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.831830 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.831958 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:40.834763 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:39481: BlockingRecv error: recv got EOF from 127.18.184.130:39481 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 02:13:41 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:41 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:41.044719 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:41.038732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:33857)
0808 02:13:41.039097 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:41.039125 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:41.039304 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:41.039592 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:41.039595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:41.039838 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:41.039937 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:41.039943 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.040722 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.040727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:41.041382 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:41.041391 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.041866 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.041870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:41.041962 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:41.042524 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:41.042535 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:41.044533 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":110,"threads_started":1}
W20260808 02:13:41.044809 29084 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:41.038940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:33857 (local address 127.18.184.190:39267)
0808 02:13:41.039097 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:41.039103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:41.039213 (+ 110us) server_negotiation.cc:407] Connection header received
0808 02:13:41.039417 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:41.039420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:41.039479 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:41.039569 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:41.040066 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.040605 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.041501 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.041735 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.041849 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:41.044676 (+ 2827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:33857: BlockingRecv error: recv got EOF from 127.18.184.131:33857 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 02:13:41 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:41 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:41.622519 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:41.616178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:51801)
0808 02:13:41.616633 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:41.616660 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:41.616762 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:41.617006 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:41.617008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:41.617262 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:41.617367 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:41.617373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.618161 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.618164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:41.618752 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:41.618761 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.619196 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.619198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:41.619283 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:41.619830 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:41.619845 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:41.622338 (+ 2493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":106,"threads_started":1}
W20260808 02:13:41.622593 29089 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:41.616271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:51801 (local address 127.18.184.190:39267)
0808 02:13:41.616725 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:41.616728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:41.616745 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:13:41.616846 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:41.616848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:41.616890 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:41.616958 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:41.617502 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.618051 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.618864 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.619094 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.619171 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:41.622466 (+ 3295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:51801: BlockingRecv error: recv got EOF from 127.18.184.129:51801 (error 108)
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":86,"threads_started":1}
Aug 08 02:13:41 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:41 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:41.841694 29090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:41.835650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:41945)
0808 02:13:41.835941 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:41.835955 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:41.836057 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:41.836299 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:41.836301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:41.836491 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:41.836591 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:41.836595 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.837491 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.837495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:41.838298 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:41.838311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.838777 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.838781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:41.838859 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:41.839429 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:41.839444 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:41.841524 (+ 2080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":79,"threads_started":1}
W20260808 02:13:41.841909 29089 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:41.835746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:41945 (local address 127.18.184.190:39267)
0808 02:13:41.835936 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:41.835940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:41.836023 (+ 83us) server_negotiation.cc:407] Connection header received
0808 02:13:41.836136 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:41.836138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:41.836182 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:41.836254 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:41.836688 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.837358 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.838429 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.838650 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.838732 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:41.841794 (+ 3062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:41945: BlockingRecv error: recv got EOF from 127.18.184.130:41945 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 08 02:13:42 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:42 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.131@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:42.051393 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:42.045661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.131:39617)
0808 02:13:42.045978 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:42.045993 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:42.046100 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:42.046392 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:42.046395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:42.046593 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:42.046716 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:42.046723 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.047475 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.047478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:42.048232 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:42.048245 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.048668 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.048671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:42.048739 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:42.049356 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:42.049368 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:42.051243 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":118,"threads_started":1}
W20260808 02:13:42.051608 29089 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:42.045709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:39617 (local address 127.18.184.190:39267)
0808 02:13:42.045842 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:42.045849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:42.046075 (+ 226us) server_negotiation.cc:407] Connection header received
0808 02:13:42.046182 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:42.046184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:42.046233 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:42.046304 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:42.046835 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.047352 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.048344 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.048565 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.048666 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:42.051446 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.131:39617: BlockingRecv error: recv got EOF from 127.18.184.131:39617 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 02:13:42 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:42 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.129@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:42.629262 29092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:42.623538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.129:52919)
0808 02:13:42.623880 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:42.623893 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:42.623978 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:42.624225 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:42.624227 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:42.624436 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:42.624534 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:42.624539 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.625327 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.625330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:42.625883 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:42.625891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.626377 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.626380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:42.626444 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:42.627041 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:42.627052 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:42.629028 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":146,"threads_started":1}
W20260808 02:13:42.629436 29093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:42.623632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:52919 (local address 127.18.184.190:39267)
0808 02:13:42.623988 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:42.623992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:42.624009 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:13:42.624050 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:42.624052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:42.624105 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:42.624191 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:42.624648 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.625228 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.626015 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.626260 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.626350 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:42.629258 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.129:52919: BlockingRecv error: recv got EOF from 127.18.184.129:52919 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":65,"threads_started":1}
Aug 08 02:13:42 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:42 dist-test-slave-9gcw krb5kdc[27951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.184.130@KRBTEST.COM for oryx/127.18.184.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:42.848814 29094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:42.842802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39267 (local address 127.18.184.130:46119)
0808 02:13:42.843108 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:42.843125 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:42.843274 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:42.843562 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:42.843565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:42.843833 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:42.843928 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:42.843935 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.844715 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.844721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:42.845316 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:42.845325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.845790 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.845794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:42.845880 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:42.846402 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:42.846413 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:42.848680 (+ 2267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.184.190:39267: Server oryx/127.18.184.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":92,"threads_started":1}
W20260808 02:13:42.849037 29093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:42.842861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:46119 (local address 127.18.184.190:39267)
0808 02:13:42.843082 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:42.843087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:42.843216 (+ 129us) server_negotiation.cc:407] Connection header received
0808 02:13:42.843382 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:42.843385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:42.843440 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:42.843535 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:42.844062 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.844603 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.845439 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.845669 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.845784 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:42.848906 (+ 3122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.184.130:46119: BlockingRecv error: recv got EOF from 127.18.184.130:46119 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
I20260808 02:13:42.930828 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 28576
I20260808 02:13:42.944643 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 28744
I20260808 02:13:42.951591 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 28878
I20260808 02:13:42.960572 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 28501
2026-08-08T02:13:42Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18299 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T02:13:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:42Z Disabled control of system clock
I20260808 02:13:42.995858 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:38251
--webserver_interface=127.18.184.190
--webserver_port=0
--builtin_ntp_servers=127.18.184.148:44575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:38251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 02:13:43.111697 29098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:43.111979 29098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:43.112016 29098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:43.116118 29098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:43.116192 29098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:43.116207 29098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:43.116225 29098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:43.116240 29098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:43.121136 29098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:38251
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:38251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:43.122354 29098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:43.123860 29098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:43.134097 29103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:43.134213 29107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:43.134155 29098 server_base.cc:1034] running on GCE node
W20260808 02:13:43.134081 29104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:43.134908 29098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:43.136258 29098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:43.137465 29098 hybrid_clock.cc:648] HybridClock initialized: now 1786155223137455 us; error 30 us; skew 500 ppm
I20260808 02:13:43.139695 29098 webserver.cc:460] Webserver started at http://127.18.184.190:39479/ using document root <none> and password file <none>
I20260808 02:13:43.140363 29098 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:43.140473 29098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:43.140753 29098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:43.143291 29098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "85476491356c47d48d286128f640f9c6"
format_stamp: "Formatted at 2026-08-08 02:13:43 on dist-test-slave-9gcw"
server_key: "b86dc7f9f9f0047036a7a8db931aabf9"
server_key_iv: "8560b72654b0d7b0da957a9330718eb5"
server_key_version: "encryptionkey@0"
I20260808 02:13:43.143939 29098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "85476491356c47d48d286128f640f9c6"
format_stamp: "Formatted at 2026-08-08 02:13:43 on dist-test-slave-9gcw"
server_key: "b86dc7f9f9f0047036a7a8db931aabf9"
server_key_iv: "8560b72654b0d7b0da957a9330718eb5"
server_key_version: "encryptionkey@0"
I20260808 02:13:43.148090 29098 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:13:43.150890 29113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:43.152199 29098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:43.152392 29098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "85476491356c47d48d286128f640f9c6"
format_stamp: "Formatted at 2026-08-08 02:13:43 on dist-test-slave-9gcw"
server_key: "b86dc7f9f9f0047036a7a8db931aabf9"
server_key_iv: "8560b72654b0d7b0da957a9330718eb5"
server_key_version: "encryptionkey@0"
I20260808 02:13:43.152522 29098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:43.167771 29098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:43.168612 29098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:43.168844 29098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:43.176779 29098 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:38251
I20260808 02:13:43.176800 29169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:38251 every 8 connection(s)
I20260808 02:13:43.177882 29098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:43.181160 29170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:43.182370 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 29098
I20260808 02:13:43.182514 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:43.182862 19170 external_mini_cluster.cc:1428] Setting key 9247edd3d3da2e5a1c8d82f1b93081d3
I20260808 02:13:43.188009 29170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6: Bootstrap starting.
I20260808 02:13:43.190708 29170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:43.191689 29170 log.cc:824] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:43.193742 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:43.184446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57758 (local address 127.18.184.190:38251)
0808 02:13:43.185132 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:43.185181 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:43.185225 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:13:43.186137 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:43.186159 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:43.186663 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:43.187033 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:43.187479 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.188901 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.189837 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.190506 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.190683 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.191553 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:43.191562 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:43.191579 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:43.192087 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:43.192162 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:43.192195 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:43.192281 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:43.192459 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 02:13:43.192743 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":121,"threads_started":1}
I20260808 02:13:43.194322 29170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6: No bootstrap required, opened a new log
I20260808 02:13:43.198388 29170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85476491356c47d48d286128f640f9c6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38251 } }
I20260808 02:13:43.198619 29170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:43.198653 29170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85476491356c47d48d286128f640f9c6, State: Initialized, Role: FOLLOWER
I20260808 02:13:43.199088 29170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85476491356c47d48d286128f640f9c6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38251 } }
I20260808 02:13:43.199225 29170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:43.199273 29170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:43.199339 29170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:43.200488 29170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85476491356c47d48d286128f640f9c6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38251 } }
I20260808 02:13:43.200825 29170 leader_election.cc:302] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85476491356c47d48d286128f640f9c6; no voters:
I20260808 02:13:43.201165 29170 leader_election.cc:288] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:43.201383 29175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:43.201731 29175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [term 1 LEADER]: Becoming Leader. State: Replica: 85476491356c47d48d286128f640f9c6, State: Running, Role: LEADER
I20260808 02:13:43.202353 29175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85476491356c47d48d286128f640f9c6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38251 } }
I20260808 02:13:43.202644 29170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:43.203927 29176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85476491356c47d48d286128f640f9c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85476491356c47d48d286128f640f9c6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38251 } } }
I20260808 02:13:43.204066 29176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:43.204434 29177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85476491356c47d48d286128f640f9c6. Latest consensus state: current_term: 1 leader_uuid: "85476491356c47d48d286128f640f9c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85476491356c47d48d286128f640f9c6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 38251 } } }
I20260808 02:13:43.204530 29177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:43.204527 29185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:43.207655 29185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:43.212848 29185 catalog_manager.cc:1347] Generated new cluster ID: 864cb0aba6f94af2876ef267866f792e
I20260808 02:13:43.212913 29185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:43.236492 29185 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:43.238013 29185 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:43.244818 29185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85476491356c47d48d286128f640f9c6: Generated new TSK 0
I20260808 02:13:43.245676 29185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:43.261819 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:38251
--builtin_ntp_servers=127.18.184.148:44575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 02:13:43.383616 29195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:43.383924 29195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:43.384013 29195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:43.388095 29195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:43.388219 29195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:43.388334 29195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:43.393211 29195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:43.394428 29195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:43.395895 29195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:43.406595 29200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:43.406631 29203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:43.406627 29201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:43.406922 29195 server_base.cc:1034] running on GCE node
I20260808 02:13:43.407353 29195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:43.408023 29195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:43.409256 29195 hybrid_clock.cc:648] HybridClock initialized: now 1786155223409217 us; error 47 us; skew 500 ppm
I20260808 02:13:43.411192 29195 webserver.cc:460] Webserver started at http://127.18.184.129:33037/ using document root <none> and password file <none>
I20260808 02:13:43.411782 29195 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:43.411877 29195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:43.412127 29195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:43.414292 29195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "eab0deef42fd4496bf2b383e9dbc1c23"
format_stamp: "Formatted at 2026-08-08 02:13:43 on dist-test-slave-9gcw"
server_key: "33985973cf17dd965a7509cb8b582217"
server_key_iv: "ff3cd117d60f51d1194c0b9f48a8954a"
server_key_version: "encryptionkey@0"
I20260808 02:13:43.414862 29195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "eab0deef42fd4496bf2b383e9dbc1c23"
format_stamp: "Formatted at 2026-08-08 02:13:43 on dist-test-slave-9gcw"
server_key: "33985973cf17dd965a7509cb8b582217"
server_key_iv: "ff3cd117d60f51d1194c0b9f48a8954a"
server_key_version: "encryptionkey@0"
I20260808 02:13:43.418711 29195 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:13:43.421409 29209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:43.422676 29195 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 02:13:43.422827 29195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "eab0deef42fd4496bf2b383e9dbc1c23"
format_stamp: "Formatted at 2026-08-08 02:13:43 on dist-test-slave-9gcw"
server_key: "33985973cf17dd965a7509cb8b582217"
server_key_iv: "ff3cd117d60f51d1194c0b9f48a8954a"
server_key_version: "encryptionkey@0"
I20260808 02:13:43.422966 29195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:43.440882 29195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:43.441735 29195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:43.441955 29195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:43.442591 29195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:43.443583 29195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:43.443660 29195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:43.443728 29195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:43.443782 29195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:43.453631 29195 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:41915
I20260808 02:13:43.453696 29327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:41915 every 8 connection(s)
I20260808 02:13:43.454659 29195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:43.460024 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 29195
I20260808 02:13:43.460197 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:43.460546 19170 external_mini_cluster.cc:1428] Setting key 19b27359e53df7bc705f23e1a172083d
I20260808 02:13:43.462917 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:38251
--builtin_ntp_servers=127.18.184.148:44575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 02:13:43.465942 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:43.456622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:42847 (local address 127.18.184.190:38251)
0808 02:13:43.456781 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:43.456785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:43.458177 (+ 1392us) server_negotiation.cc:407] Connection header received
0808 02:13:43.459585 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:43.459588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:43.459644 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:43.459744 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:43.461124 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.461736 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.463513 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.463756 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.463838 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.465350 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:43.465353 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:43.465356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:43.465371 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:43.465401 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:43.465409 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:43.465492 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:43.465665 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 02:13:43.465785 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:13:43.466881 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:43.457071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:38251 (local address 127.18.184.129:42847)
0808 02:13:43.458007 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:43.458055 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:43.459391 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:43.459967 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:43.459978 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:43.460092 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:43.460940 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:43.460963 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.461888 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.461891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:43.463358 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:43.463373 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.463924 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.463930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:43.464171 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.464969 (+ 798us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:43.464972 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:43.465040 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:43.465116 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:43.465494 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:43.465507 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:43.465514 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:13:43.465673 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 02:13:43.465989 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":123,"threads_started":1}
I20260808 02:13:43.468364 29328 heartbeater.cc:347] Connected to a master server at 127.18.184.190:38251
I20260808 02:13:43.468669 29328 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:43.469463 29328 heartbeater.cc:511] Master 127.18.184.190:38251 requested a full tablet report, sending...
I20260808 02:13:43.471688 29131 ts_manager.cc:194] Registered new tserver with Master: eab0deef42fd4496bf2b383e9dbc1c23 (127.18.184.129:41915)
I20260808 02:13:43.473604 29131 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.184.129:42847
W20260808 02:13:43.581338 29331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:43.581593 29331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:43.581630 29331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:43.586359 29331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:43.586421 29331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:43.586480 29331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:43.591176 29331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:43.592222 29331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:43.593657 29331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:43.604290 29336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:43.604385 29331 server_base.cc:1034] running on GCE node
W20260808 02:13:43.604393 29337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:43.604578 29339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:43.605079 29331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:43.605849 29331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:43.607039 29331 hybrid_clock.cc:648] HybridClock initialized: now 1786155223607017 us; error 30 us; skew 500 ppm
I20260808 02:13:43.608952 29331 webserver.cc:460] Webserver started at http://127.18.184.130:42313/ using document root <none> and password file <none>
I20260808 02:13:43.609602 29331 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:43.609668 29331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:43.609920 29331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:43.612102 29331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "7e3fad618c744c6997287e2f2a26336d"
format_stamp: "Formatted at 2026-08-08 02:13:43 on dist-test-slave-9gcw"
server_key: "1bcdcdfa2ccbdc4e0123ca97f744ed19"
server_key_iv: "eee4ba81f2113f19f364792779145665"
server_key_version: "encryptionkey@0"
I20260808 02:13:43.612727 29331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "7e3fad618c744c6997287e2f2a26336d"
format_stamp: "Formatted at 2026-08-08 02:13:43 on dist-test-slave-9gcw"
server_key: "1bcdcdfa2ccbdc4e0123ca97f744ed19"
server_key_iv: "eee4ba81f2113f19f364792779145665"
server_key_version: "encryptionkey@0"
I20260808 02:13:43.616639 29331 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 02:13:43.619565 29345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:43.621309 29331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:43.621445 29331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "7e3fad618c744c6997287e2f2a26336d"
format_stamp: "Formatted at 2026-08-08 02:13:43 on dist-test-slave-9gcw"
server_key: "1bcdcdfa2ccbdc4e0123ca97f744ed19"
server_key_iv: "eee4ba81f2113f19f364792779145665"
server_key_version: "encryptionkey@0"
I20260808 02:13:43.621583 29331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:43.658859 29331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:43.659618 29331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:43.659794 29331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:43.660380 29331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:43.661396 29331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:43.661448 29331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:43.661536 29331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:43.661571 29331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:43.671236 29331 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:45963
I20260808 02:13:43.671342 29470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:45963 every 8 connection(s)
I20260808 02:13:43.672238 29331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:43.681370 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:43.674211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:39615 (local address 127.18.184.190:38251)
0808 02:13:43.674367 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:43.674372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:43.675498 (+ 1126us) server_negotiation.cc:407] Connection header received
0808 02:13:43.676836 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:43.676839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:43.676888 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:43.676964 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:43.677811 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.678386 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.679512 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.679720 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.679810 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.680790 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:43.680793 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:43.680795 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:43.680807 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:43.680827 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:43.680835 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:43.680892 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:43.681122 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 02:13:43.681247 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:13:43.682085 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 29331
I20260808 02:13:43.682204 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:43.682253 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:43.674659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:38251 (local address 127.18.184.130:39615)
0808 02:13:43.675336 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:43.675391 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:43.676631 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:43.677110 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:43.677117 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:43.677213 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:43.677675 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:43.677686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.678511 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.678514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:43.679400 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:43.679410 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.679860 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.679862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:43.680006 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.680553 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:43.680554 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:43.680591 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:43.680649 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:43.680910 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:43.680917 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:43.680922 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:13:43.681015 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 02:13:43.681302 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":124,"threads_started":1}
I20260808 02:13:43.682548 19170 external_mini_cluster.cc:1428] Setting key 31e7e7d006e1f6642b09e0bddd6ec733
I20260808 02:13:43.683534 29471 heartbeater.cc:347] Connected to a master server at 127.18.184.190:38251
I20260808 02:13:43.683809 29471 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:43.684415 29471 heartbeater.cc:511] Master 127.18.184.190:38251 requested a full tablet report, sending...
I20260808 02:13:43.684649 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:38251
--builtin_ntp_servers=127.18.184.148:44575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 02:13:43.685511 29131 ts_manager.cc:194] Registered new tserver with Master: 7e3fad618c744c6997287e2f2a26336d (127.18.184.130:45963)
I20260808 02:13:43.686220 29131 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.184.130:39615
W20260808 02:13:43.803201 29474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:43.803465 29474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:43.803529 29474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:43.807855 29474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:43.807947 29474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:43.808045 29474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:43.813215 29474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:43.814455 29474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:43.815930 29474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:43.825871 29480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:43.825910 29483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:43.825950 29481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:43.826265 29474 server_base.cc:1034] running on GCE node
I20260808 02:13:43.826900 29474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:43.827766 29474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:43.828969 29474 hybrid_clock.cc:648] HybridClock initialized: now 1786155223828933 us; error 45 us; skew 500 ppm
I20260808 02:13:43.830958 29474 webserver.cc:460] Webserver started at http://127.18.184.131:36039/ using document root <none> and password file <none>
I20260808 02:13:43.831554 29474 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:43.831624 29474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:43.831872 29474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:43.834071 29474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "c1fa2ac592c243bcb4cbfc6706521f05"
format_stamp: "Formatted at 2026-08-08 02:13:43 on dist-test-slave-9gcw"
server_key: "31c2b27c5d05d8e9e712cc464a1081a8"
server_key_iv: "51799cb5977fb5ce0c0bab0e4b2cb3b8"
server_key_version: "encryptionkey@0"
I20260808 02:13:43.834646 29474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "c1fa2ac592c243bcb4cbfc6706521f05"
format_stamp: "Formatted at 2026-08-08 02:13:43 on dist-test-slave-9gcw"
server_key: "31c2b27c5d05d8e9e712cc464a1081a8"
server_key_iv: "51799cb5977fb5ce0c0bab0e4b2cb3b8"
server_key_version: "encryptionkey@0"
I20260808 02:13:43.838330 29474 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:13:43.840852 29490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:43.842177 29474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:43.842336 29474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "c1fa2ac592c243bcb4cbfc6706521f05"
format_stamp: "Formatted at 2026-08-08 02:13:43 on dist-test-slave-9gcw"
server_key: "31c2b27c5d05d8e9e712cc464a1081a8"
server_key_iv: "51799cb5977fb5ce0c0bab0e4b2cb3b8"
server_key_version: "encryptionkey@0"
I20260808 02:13:43.842455 29474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:43.873242 29474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:43.874010 29474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:43.874229 29474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:43.874872 29474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:43.875902 29474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:43.875976 29474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:43.876046 29474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:43.876099 29474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:43.885073 29474 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:38535
I20260808 02:13:43.885105 29613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:38535 every 8 connection(s)
I20260808 02:13:43.886121 29474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:43.892534 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 29474
I20260808 02:13:43.892668 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:43.893015 19170 external_mini_cluster.cc:1428] Setting key 1be89856772ff2c3cd38e66c603aab82
I20260808 02:13:43.895831 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:43.888044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:58085 (local address 127.18.184.190:38251)
0808 02:13:43.888195 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:43.888199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:43.889294 (+ 1095us) server_negotiation.cc:407] Connection header received
0808 02:13:43.890704 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:43.890707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:43.890777 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:43.890883 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:43.891788 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.892323 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.893764 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.893985 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.894119 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.895444 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:43.895447 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:43.895449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:43.895460 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:43.895480 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:43.895488 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:43.895548 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:43.895664 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 02:13:43.895722 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:13:43.896749 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:43.888453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:38251 (local address 127.18.184.131:58085)
0808 02:13:43.889083 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:43.889123 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:43.890490 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:43.891056 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:43.891062 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:43.891146 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:43.891635 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:43.891648 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.892454 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.892462 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:43.893637 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:43.893647 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.894191 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.894196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:43.894455 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.895163 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:43.895165 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:43.895204 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:43.895300 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:43.895564 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:43.895571 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:43.895575 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:43.895667 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 02:13:43.895913 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":106,"threads_started":1}
I20260808 02:13:43.898041 29614 heartbeater.cc:347] Connected to a master server at 127.18.184.190:38251
I20260808 02:13:43.898319 29614 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:43.898845 29614 heartbeater.cc:511] Master 127.18.184.190:38251 requested a full tablet report, sending...
I20260808 02:13:43.899930 29131 ts_manager.cc:194] Registered new tserver with Master: c1fa2ac592c243bcb4cbfc6706521f05 (127.18.184.131:38535)
I20260808 02:13:43.900532 29131 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.184.131:58085
I20260808 02:13:43.907135 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:13:43.910929 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 29195
W20260808 02:13:43.919817 29173 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:43.909829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57762 (local address 127.18.184.190:38251)
0808 02:13:43.910158 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:43.910165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:43.910182 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:13:43.910222 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:43.910225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:43.910282 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:43.910366 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:43.919612 (+ 9246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57762: BlockingRecv error: recv got EOF from 127.0.0.1:57762 (error 108)
Metrics: {"server-negotiator.queue_time_us":183}
I20260808 02:13:43.925083 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 29331
I20260808 02:13:43.933068 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 29474
I20260808 02:13:43.939770 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 29098
2026-08-08T02:13:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (976 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T02:13:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:43Z Disabled control of system clock
I20260808 02:13:43.973588 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:40747
--webserver_interface=127.18.184.190
--webserver_port=0
--builtin_ntp_servers=127.18.184.148:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:40747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:13:44.089538 29625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:44.089830 29625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:44.089869 29625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:44.093927 29625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:44.093988 29625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:44.094003 29625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:44.094019 29625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:44.094035 29625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:44.098793 29625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:40747
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:40747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:44.100132 29625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:44.101637 29625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:44.110562 29631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:44.110682 29625 server_base.cc:1034] running on GCE node
W20260808 02:13:44.110561 29630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:44.110644 29633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:44.111426 29625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:44.112694 29625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:44.113917 29625 hybrid_clock.cc:648] HybridClock initialized: now 1786155224113890 us; error 45 us; skew 500 ppm
I20260808 02:13:44.116175 29625 webserver.cc:460] Webserver started at http://127.18.184.190:42233/ using document root <none> and password file <none>
I20260808 02:13:44.116845 29625 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:44.116956 29625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:44.117298 29625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:44.119908 29625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "2b984ea5d30e4c248f4ced84c6bae4ab"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-9gcw"
server_key: "37cc26bf57e945f1fc621cf51bb75f2e"
server_key_iv: "b0b669a1fc3be40c21d87f6e872cf071"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.120553 29625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "2b984ea5d30e4c248f4ced84c6bae4ab"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-9gcw"
server_key: "37cc26bf57e945f1fc621cf51bb75f2e"
server_key_iv: "b0b669a1fc3be40c21d87f6e872cf071"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.124954 29625 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 02:13:44.127703 29639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:44.129072 29625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:44.129280 29625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "2b984ea5d30e4c248f4ced84c6bae4ab"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-9gcw"
server_key: "37cc26bf57e945f1fc621cf51bb75f2e"
server_key_iv: "b0b669a1fc3be40c21d87f6e872cf071"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.129415 29625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:44.134310 29625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:44.134968 29625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:44.135164 29625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:44.142827 29625 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:40747
I20260808 02:13:44.142851 29693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:40747 every 8 connection(s)
I20260808 02:13:44.143851 29625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:44.147220 29694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:44.150573 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 29625
I20260808 02:13:44.150759 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:44.151081 19170 external_mini_cluster.cc:1428] Setting key 1de60c957dc36fdbd64836df319d7504
I20260808 02:13:44.153436 29694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab: Bootstrap starting.
I20260808 02:13:44.156014 29694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:44.156778 29694 log.cc:824] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:44.157058 29698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:44.152587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.18.184.190:40747)
0808 02:13:44.153303 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:44.153321 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:44.153360 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:13:44.154107 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:44.154132 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:44.154484 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:44.154770 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:44.155636 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:44.155641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:44.155652 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:44.155663 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:44.155722 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:44.155737 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:44.155825 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:44.155981 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 02:13:44.156294 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":130,"threads_started":1}
I20260808 02:13:44.159169 29694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab: No bootstrap required, opened a new log
I20260808 02:13:44.163969 29694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b984ea5d30e4c248f4ced84c6bae4ab" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40747 } }
I20260808 02:13:44.164240 29694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:44.164348 29694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b984ea5d30e4c248f4ced84c6bae4ab, State: Initialized, Role: FOLLOWER
I20260808 02:13:44.164813 29694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b984ea5d30e4c248f4ced84c6bae4ab" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40747 } }
I20260808 02:13:44.164985 29694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:44.165067 29694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:44.165222 29694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:44.166399 29694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b984ea5d30e4c248f4ced84c6bae4ab" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40747 } }
I20260808 02:13:44.166744 29694 leader_election.cc:302] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b984ea5d30e4c248f4ced84c6bae4ab; no voters:
I20260808 02:13:44.167150 29694 leader_election.cc:288] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:44.167331 29701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:44.167595 29701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [term 1 LEADER]: Becoming Leader. State: Replica: 2b984ea5d30e4c248f4ced84c6bae4ab, State: Running, Role: LEADER
I20260808 02:13:44.168002 29701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b984ea5d30e4c248f4ced84c6bae4ab" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40747 } }
I20260808 02:13:44.168666 29694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:44.169858 29703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b984ea5d30e4c248f4ced84c6bae4ab. Latest consensus state: current_term: 1 leader_uuid: "2b984ea5d30e4c248f4ced84c6bae4ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b984ea5d30e4c248f4ced84c6bae4ab" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40747 } } }
I20260808 02:13:44.170043 29703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:44.170395 29702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b984ea5d30e4c248f4ced84c6bae4ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b984ea5d30e4c248f4ced84c6bae4ab" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40747 } } }
I20260808 02:13:44.170538 29702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:44.171598 29710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:44.174389 29710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:44.181030 29710 catalog_manager.cc:1347] Generated new cluster ID: ddbc54c8e54446aeb616be5c9add6edb
I20260808 02:13:44.181107 29710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:44.199175 29710 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:44.199556 29710 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:44.215384 29710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b984ea5d30e4c248f4ced84c6bae4ab: Generated new TSK 0
I20260808 02:13:44.216233 29710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:44.224370 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40747
--builtin_ntp_servers=127.18.184.148:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:13:44.363994 29722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:44.364275 29722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:44.364319 29722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:44.368477 29722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:44.368552 29722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:44.368623 29722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:44.373761 29722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:44.374861 29722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:44.376308 29722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:44.387336 29727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:44.387337 29728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:44.387336 29731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:44.387620 29722 server_base.cc:1034] running on GCE node
I20260808 02:13:44.388382 29722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:44.389038 29722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:44.390232 29722 hybrid_clock.cc:648] HybridClock initialized: now 1786155224390209 us; error 59 us; skew 500 ppm
I20260808 02:13:44.392436 29722 webserver.cc:460] Webserver started at http://127.18.184.129:37065/ using document root <none> and password file <none>
I20260808 02:13:44.393117 29722 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:44.393280 29722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:44.393558 29722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:44.395713 29722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "9dcb3415a9e3445e8b2dd2620efa51d2"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-9gcw"
server_key: "e16cb40042ff02b4b20bfda4c50241a8"
server_key_iv: "42a6308155b518a2aef96fe3db179250"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.396301 29722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "9dcb3415a9e3445e8b2dd2620efa51d2"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-9gcw"
server_key: "e16cb40042ff02b4b20bfda4c50241a8"
server_key_iv: "42a6308155b518a2aef96fe3db179250"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.400326 29722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:13:44.403400 29737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:44.404865 29722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:13:44.405033 29722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "9dcb3415a9e3445e8b2dd2620efa51d2"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-9gcw"
server_key: "e16cb40042ff02b4b20bfda4c50241a8"
server_key_iv: "42a6308155b518a2aef96fe3db179250"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.405187 29722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:44.410029 29722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:44.410681 29722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:44.410888 29722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:44.411469 29722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:44.412410 29722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:44.412484 29722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:44.412556 29722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:44.412601 29722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:44.422513 29722 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:46325
I20260808 02:13:44.422530 29860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:46325 every 8 connection(s)
I20260808 02:13:44.423647 29722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:44.430570 29698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:44.425769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:46769 (local address 127.18.184.190:40747)
0808 02:13:44.425938 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:44.425943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:44.427240 (+ 1297us) server_negotiation.cc:407] Connection header received
0808 02:13:44.428588 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:44.428591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:44.428632 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:44.428721 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:44.430112 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:44.430116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:44.430118 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:44.430132 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:44.430155 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:44.430164 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:44.430286 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:44.430400 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 02:13:44.430445 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:13:44.431555 29863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:44.426333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40747 (local address 127.18.184.129:46769)
0808 02:13:44.427040 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:44.427086 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:44.428383 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:44.428967 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:44.428974 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:44.429070 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:44.429825 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:44.429827 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:44.429870 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:44.429943 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:44.430257 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:44.430267 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:44.430272 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:13:44.430423 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 02:13:44.430667 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":116,"threads_started":1}
I20260808 02:13:44.433059 29861 heartbeater.cc:347] Connected to a master server at 127.18.184.190:40747
I20260808 02:13:44.433336 29861 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:44.433583 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 29722
I20260808 02:13:44.433775 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:44.433887 29861 heartbeater.cc:511] Master 127.18.184.190:40747 requested a full tablet report, sending...
I20260808 02:13:44.434151 19170 external_mini_cluster.cc:1428] Setting key cb469e2a68d5289e9821d78eef286b82
I20260808 02:13:44.435596 29656 ts_manager.cc:194] Registered new tserver with Master: 9dcb3415a9e3445e8b2dd2620efa51d2 (127.18.184.129:46325)
I20260808 02:13:44.436563 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40747
--builtin_ntp_servers=127.18.184.148:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:13:44.557767 29865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:44.558069 29865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:44.558115 29865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:44.562507 29865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:44.562604 29865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:44.562669 29865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:44.567723 29865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:44.568959 29865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:44.570609 29865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:44.581373 29873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:44.581422 29872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:44.581440 29875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:44.581598 29865 server_base.cc:1034] running on GCE node
I20260808 02:13:44.582155 29865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:44.582875 29865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:44.584106 29865 hybrid_clock.cc:648] HybridClock initialized: now 1786155224584068 us; error 46 us; skew 500 ppm
I20260808 02:13:44.586228 29865 webserver.cc:460] Webserver started at http://127.18.184.130:43751/ using document root <none> and password file <none>
I20260808 02:13:44.586829 29865 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:44.586923 29865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:44.587159 29865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:44.589345 29865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "4c4db556be7c4a7294c9078c7b53fd65"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-9gcw"
server_key: "646647756f2ef9c1e5ac14776a72df95"
server_key_iv: "e8770456d5e1530d873e0fbbc1f177c2"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.589946 29865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "4c4db556be7c4a7294c9078c7b53fd65"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-9gcw"
server_key: "646647756f2ef9c1e5ac14776a72df95"
server_key_iv: "e8770456d5e1530d873e0fbbc1f177c2"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.593977 29865 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 02:13:44.596807 29882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:44.598215 29865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:13:44.598352 29865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "4c4db556be7c4a7294c9078c7b53fd65"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-9gcw"
server_key: "646647756f2ef9c1e5ac14776a72df95"
server_key_iv: "e8770456d5e1530d873e0fbbc1f177c2"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.598479 29865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:44.603961 29865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:44.604694 29865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:44.604907 29865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:44.605661 29865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:44.606686 29865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:44.606762 29865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:44.606837 29865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:44.606882 29865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:44.617774 29865 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:42279
I20260808 02:13:44.617805 30007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:42279 every 8 connection(s)
I20260808 02:13:44.618884 29865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:44.624791 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 29865
I20260808 02:13:44.624989 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:44.625375 19170 external_mini_cluster.cc:1428] Setting key 4e4c6d5f4504d3ebcf863e5d4058f5bf
I20260808 02:13:44.626122 29698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:44.620868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:49291 (local address 127.18.184.190:40747)
0808 02:13:44.621023 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:44.621027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:44.622385 (+ 1358us) server_negotiation.cc:407] Connection header received
0808 02:13:44.623846 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:44.623849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:44.623892 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:44.623978 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:44.625514 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:44.625521 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:44.625524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:44.625540 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:44.625566 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:44.625582 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:44.625673 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:44.625880 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 02:13:44.625938 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:13:44.627226 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:44.621427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40747 (local address 127.18.184.130:49291)
0808 02:13:44.622212 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:44.622257 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:44.623593 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:44.624173 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:44.624180 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:44.624273 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:44.625079 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:44.625082 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:44.625174 (+ 92us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:44.625279 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:44.625665 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:44.625677 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:44.625683 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:13:44.625886 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 02:13:44.626156 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":144,"threads_started":1}
I20260808 02:13:44.627925 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40747
--builtin_ntp_servers=127.18.184.148:38845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 02:13:44.628674 30009 heartbeater.cc:347] Connected to a master server at 127.18.184.190:40747
I20260808 02:13:44.628968 30009 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:44.629557 30009 heartbeater.cc:511] Master 127.18.184.190:40747 requested a full tablet report, sending...
I20260808 02:13:44.631014 29656 ts_manager.cc:194] Registered new tserver with Master: 4c4db556be7c4a7294c9078c7b53fd65 (127.18.184.130:42279)
W20260808 02:13:44.750478 30013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:44.750837 30013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:44.750921 30013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:44.755203 30013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:44.755311 30013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:44.755425 30013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:44.760578 30013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:38845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:44.761853 30013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:44.763361 30013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:44.774187 30019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:44.774210 30018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:44.774150 30021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:44.774705 30013 server_base.cc:1034] running on GCE node
I20260808 02:13:44.775138 30013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:44.775825 30013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:44.777024 30013 hybrid_clock.cc:648] HybridClock initialized: now 1786155224776994 us; error 42 us; skew 500 ppm
I20260808 02:13:44.779222 30013 webserver.cc:460] Webserver started at http://127.18.184.131:32821/ using document root <none> and password file <none>
I20260808 02:13:44.779832 30013 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:44.779930 30013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:44.780172 30013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:44.782325 30013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "2551ceeda22f4a10b69be6f14c9a6201"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-9gcw"
server_key: "9a936b26105caddbc2cb43ae4e581060"
server_key_iv: "e057541ac87f808c11d5bd1040153ce8"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.782914 30013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "2551ceeda22f4a10b69be6f14c9a6201"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-9gcw"
server_key: "9a936b26105caddbc2cb43ae4e581060"
server_key_iv: "e057541ac87f808c11d5bd1040153ce8"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.787032 30013 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 02:13:44.790120 30028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:44.791571 30013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:44.791797 30013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "2551ceeda22f4a10b69be6f14c9a6201"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-9gcw"
server_key: "9a936b26105caddbc2cb43ae4e581060"
server_key_iv: "e057541ac87f808c11d5bd1040153ce8"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.791941 30013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:44.797220 30013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:44.797915 30013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:44.798133 30013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:44.798755 30013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:44.799767 30013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:44.799845 30013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:44.799925 30013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:44.799975 30013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:44.810232 30013 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:36135
I20260808 02:13:44.810297 30148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:36135 every 8 connection(s)
I20260808 02:13:44.811314 30013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:44.816869 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 30013
I20260808 02:13:44.817006 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:44.817394 19170 external_mini_cluster.cc:1428] Setting key b0b9410c3a7687f1e8e1698464723a4a
I20260808 02:13:44.818538 29698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:44.813549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:55557 (local address 127.18.184.190:40747)
0808 02:13:44.813734 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:44.813743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:44.814949 (+ 1206us) server_negotiation.cc:407] Connection header received
0808 02:13:44.816277 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:44.816280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:44.816322 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:44.816408 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:44.817849 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:44.817854 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:44.817858 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:44.817875 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:44.817903 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:44.817919 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:44.818030 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:44.818311 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 02:13:44.818373 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 02:13:44.819460 30153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:44.814028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40747 (local address 127.18.184.131:55557)
0808 02:13:44.814783 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:44.814829 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:44.816063 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:44.816619 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:44.816629 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:44.816806 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:44.817511 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:44.817513 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:44.817568 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:44.817653 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:44.818022 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:44.818031 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:44.818036 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:13:44.818213 (+ 177us) client_negotiation.cc:240] Negotiation successful
0808 02:13:44.818495 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":118,"threads_started":1}
I20260808 02:13:44.820592 30151 heartbeater.cc:347] Connected to a master server at 127.18.184.190:40747
I20260808 02:13:44.820847 30151 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:44.821426 30151 heartbeater.cc:511] Master 127.18.184.190:40747 requested a full tablet report, sending...
I20260808 02:13:44.822685 29656 ts_manager.cc:194] Registered new tserver with Master: 2551ceeda22f4a10b69be6f14c9a6201 (127.18.184.131:36135)
I20260808 02:13:44.832394 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:13:44.836702 29698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:44.835731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43276 (local address 127.18.184.190:40747)
0808 02:13:44.835896 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:44.835903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:44.835930 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:13:44.835978 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:44.835984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:44.836033 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:44.836135 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:44.836514 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43276: BlockingRecv error: recv got EOF from 127.0.0.1:43276 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:13:44.837319 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 29722
I20260808 02:13:44.859272 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 29865
I20260808 02:13:44.876116 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 30013
I20260808 02:13:44.882457 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 29625
2026-08-08T02:13:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (944 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:44 dist-test-slave-9gcw krb5kdc[30162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:44 dist-test-slave-9gcw krb5kdc[30162](info): set up 2 sockets
Aug 08 02:13:44 dist-test-slave-9gcw krb5kdc[30162](info): commencing operation
krb5kdc: starting...
W20260808 02:13:45.985560 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:13:46 dist-test-slave-9gcw krb5kdc[30162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:13:46.125536 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:36727
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:36727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:46.246199 30175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:46.246510 30175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:46.246553 30175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:46.250854 30175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:46.250934 30175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:46.250953 30175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:46.250972 30175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:46.250986 30175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:46.256712 30175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:36727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:36727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:46.257949 30175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:46.259068 30175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:46.268272 30183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:46.268270 30182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:46.268373 30175 server_base.cc:1034] running on GCE node
W20260808 02:13:46.268270 30185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:46.269189 30175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:46.270419 30175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:46.271620 30175 hybrid_clock.cc:648] HybridClock initialized: now 1786155226271608 us; error 35 us; skew 500 ppm
Aug 08 02:13:46 dist-test-slave-9gcw krb5kdc[30162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155226, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:46.274751 30175 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:13:46.275801 30175 webserver.cc:460] Webserver started at http://127.18.184.190:37063/ using document root <none> and password file <none>
I20260808 02:13:46.276423 30175 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:46.276521 30175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:46.276777 30175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:46.279009 30175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "3b88f9b51a5c4a418cf3bbc5d51a87b3"
format_stamp: "Formatted at 2026-08-08 02:13:46 on dist-test-slave-9gcw"
server_key: "9c517420d26da2495d0396315343ef8c"
server_key_iv: "c6b7f45f5eefbea3099ddb7d0b9820cd"
server_key_version: "encryptionkey@0"
I20260808 02:13:46.279611 30175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "3b88f9b51a5c4a418cf3bbc5d51a87b3"
format_stamp: "Formatted at 2026-08-08 02:13:46 on dist-test-slave-9gcw"
server_key: "9c517420d26da2495d0396315343ef8c"
server_key_iv: "c6b7f45f5eefbea3099ddb7d0b9820cd"
server_key_version: "encryptionkey@0"
I20260808 02:13:46.283854 30175 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:13:46.286629 30193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:46.288173 30175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:13:46.288370 30175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "3b88f9b51a5c4a418cf3bbc5d51a87b3"
format_stamp: "Formatted at 2026-08-08 02:13:46 on dist-test-slave-9gcw"
server_key: "9c517420d26da2495d0396315343ef8c"
server_key_iv: "c6b7f45f5eefbea3099ddb7d0b9820cd"
server_key_version: "encryptionkey@0"
I20260808 02:13:46.288511 30175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:46.345873 30175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:46.349924 30175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:46.350170 30175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:46.358217 30175 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:36727
I20260808 02:13:46.358534 30249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:36727 every 8 connection(s)
I20260808 02:13:46.359400 30175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:46.362380 30250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:46.364697 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 30175
I20260808 02:13:46.364805 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:13:46.365118 19170 external_mini_cluster.cc:1428] Setting key b67b5e0af84788637729bc1b7969c5a6
I20260808 02:13:46.368479 30250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3: Bootstrap starting.
I20260808 02:13:46.370769 30250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:46.371742 30250 log.cc:824] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:46.374123 30250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3: No bootstrap required, opened a new log
Aug 08 02:13:46 dist-test-slave-9gcw krb5kdc[30162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:46.377641 30250 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b88f9b51a5c4a418cf3bbc5d51a87b3" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36727 } }
I20260808 02:13:46.377957 30250 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:46.378053 30250 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b88f9b51a5c4a418cf3bbc5d51a87b3, State: Initialized, Role: FOLLOWER
I20260808 02:13:46.378554 30250 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b88f9b51a5c4a418cf3bbc5d51a87b3" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36727 } }
I20260808 02:13:46.378712 30250 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:46.378803 30250 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:46.378926 30250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:46.380172 30250 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b88f9b51a5c4a418cf3bbc5d51a87b3" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36727 } }
I20260808 02:13:46.380574 30250 leader_election.cc:302] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b88f9b51a5c4a418cf3bbc5d51a87b3; no voters:
I20260808 02:13:46.380919 30250 leader_election.cc:288] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:46.381042 30256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:46.381377 30256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [term 1 LEADER]: Becoming Leader. State: Replica: 3b88f9b51a5c4a418cf3bbc5d51a87b3, State: Running, Role: LEADER
I20260808 02:13:46.381711 30256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b88f9b51a5c4a418cf3bbc5d51a87b3" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36727 } }
I20260808 02:13:46.381822 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:46.366790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39048 (local address 127.18.184.190:36727)
0808 02:13:46.367259 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:46.367276 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:46.367313 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:13:46.367972 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:46.367989 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:46.368397 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:46.368775 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:46.369751 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:46.370830 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:46.372063 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:46.372735 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:46.372871 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:46.375721 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:46.375726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:46.375738 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:46.375765 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:46.378420 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:46.378933 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:46.378942 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:46.378948 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:46.379105 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:46.379392 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:46.379395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:46.379397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:46.379946 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:46.380167 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:46.380383 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 02:13:46.380699 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":127,"threads_started":1}
I20260808 02:13:46.382277 30250 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:46.382951 30256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b88f9b51a5c4a418cf3bbc5d51a87b3. Latest consensus state: current_term: 1 leader_uuid: "3b88f9b51a5c4a418cf3bbc5d51a87b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b88f9b51a5c4a418cf3bbc5d51a87b3" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36727 } } }
I20260808 02:13:46.383072 30256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:46.383427 30258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b88f9b51a5c4a418cf3bbc5d51a87b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b88f9b51a5c4a418cf3bbc5d51a87b3" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36727 } } }
I20260808 02:13:46.383512 30258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:46.384945 30263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:46.387796 30263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:46.393260 30263 catalog_manager.cc:1347] Generated new cluster ID: 68a6e1f95d4d437d882beb3a301e5f3d
I20260808 02:13:46.393359 30263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:46.409898 30263 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:46.411165 30263 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:46.419544 30263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3: Generated new TSK 0
I20260808 02:13:46.420336 30263 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:13:46.453480 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36727
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:46.572443 30278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:46.572755 30278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:46.572803 30278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:46.577069 30278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:46.577193 30278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:46.577298 30278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:13:46.583175 30278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:46.584400 30278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:46.585572 30278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:46.596222 30284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:46.596222 30287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:46.596303 30285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:46.596545 30278 server_base.cc:1034] running on GCE node
I20260808 02:13:46.597052 30278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:46.597786 30278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:46.599020 30278 hybrid_clock.cc:648] HybridClock initialized: now 1786155226598999 us; error 30 us; skew 500 ppm
Aug 08 02:13:46 dist-test-slave-9gcw krb5kdc[30162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155226, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:46.602128 30278 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:13:46.603082 30278 webserver.cc:460] Webserver started at http://127.18.184.129:41257/ using document root <none> and password file <none>
I20260808 02:13:46.603710 30278 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:46.603806 30278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:46.604068 30278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:46.606172 30278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "a7f01e4714bd426e98652c00312b23d9"
format_stamp: "Formatted at 2026-08-08 02:13:46 on dist-test-slave-9gcw"
server_key: "ba8cad3550023ac767f53d64c5587848"
server_key_iv: "f126bd28ef92a78c80b953a9878ada8a"
server_key_version: "encryptionkey@0"
I20260808 02:13:46.606701 30278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "a7f01e4714bd426e98652c00312b23d9"
format_stamp: "Formatted at 2026-08-08 02:13:46 on dist-test-slave-9gcw"
server_key: "ba8cad3550023ac767f53d64c5587848"
server_key_iv: "f126bd28ef92a78c80b953a9878ada8a"
server_key_version: "encryptionkey@0"
I20260808 02:13:46.610502 30278 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:13:46.613006 30296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:46.614298 30278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 02:13:46.614400 30278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "a7f01e4714bd426e98652c00312b23d9"
format_stamp: "Formatted at 2026-08-08 02:13:46 on dist-test-slave-9gcw"
server_key: "ba8cad3550023ac767f53d64c5587848"
server_key_iv: "f126bd28ef92a78c80b953a9878ada8a"
server_key_version: "encryptionkey@0"
I20260808 02:13:46.614498 30278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:46.632628 30278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:46.635852 30278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:46.636022 30278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:46.636586 30278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:46.637492 30278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:46.637542 30278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:46.637583 30278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:46.637598 30278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:46.647014 30278 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:41457
I20260808 02:13:46.647034 30425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:41457 every 8 connection(s)
I20260808 02:13:46.647996 30278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:46.651620 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 30278
I20260808 02:13:46.651760 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:46.652086 19170 external_mini_cluster.cc:1428] Setting key 90a6871f7a2810ed4ddf174eef725262
Aug 08 02:13:46 dist-test-slave-9gcw krb5kdc[30162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155226, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:13:46.663228 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:46.650004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:60179 (local address 127.18.184.190:36727)
0808 02:13:46.650170 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:46.650174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:46.651053 (+ 879us) server_negotiation.cc:407] Connection header received
0808 02:13:46.652753 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:46.652757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:46.652820 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:46.652927 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:46.654350 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:46.654896 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:46.656479 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:46.656708 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:46.656802 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:46.659530 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:46.659534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:46.659536 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:46.659562 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:46.661014 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:46.661579 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:46.661583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:46.661585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:46.661673 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:46.662041 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:46.662044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:46.662046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:46.662234 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:46.662372 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:46.662913 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 02:13:46.663045 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:13:46.664090 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:46.650389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:36727 (local address 127.18.184.129:60179)
0808 02:13:46.650887 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:46.650935 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:46.652524 (+ 1589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:46.653113 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:46.653120 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:46.653659 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:46.654183 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:46.654197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:46.655033 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:46.655039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:46.656332 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:46.656346 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:46.656880 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:46.656885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:46.657097 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:46.657790 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:46.657806 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:46.659325 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:46.661138 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:46.661170 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:46.661188 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:46.661468 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:46.661774 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:46.661778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:46.661780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:46.661936 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:46.662388 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:46.662396 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:46.662647 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 02:13:46.662855 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 02:13:46.663191 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":113,"threads_started":1}
I20260808 02:13:46.665481 30426 heartbeater.cc:347] Connected to a master server at 127.18.184.190:36727
I20260808 02:13:46.665750 30426 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
I20260808 02:13:46.666363 30426 heartbeater.cc:511] Master 127.18.184.190:36727 requested a full tablet report, sending...
I20260808 02:13:46.668090 30211 ts_manager.cc:194] Registered new tserver with Master: a7f01e4714bd426e98652c00312b23d9 (127.18.184.129:41457)
I20260808 02:13:46.669375 30211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:60179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:13:46.684926 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36727
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:46.801874 30432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:46.802174 30432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:46.802266 30432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:46.806461 30432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:46.806555 30432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:46.806646 30432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:13:46.812266 30432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:46.813551 30432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:46.814637 30432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:46.824993 30441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:46.825022 30439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:46.825129 30432 server_base.cc:1034] running on GCE node
W20260808 02:13:46.824997 30438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:46.825845 30432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:46.826490 30432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:46.827682 30432 hybrid_clock.cc:648] HybridClock initialized: now 1786155226827655 us; error 39 us; skew 500 ppm
Aug 08 02:13:46 dist-test-slave-9gcw krb5kdc[30162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155226, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:46.830500 30432 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:13:46.831382 30432 webserver.cc:460] Webserver started at http://127.18.184.130:44255/ using document root <none> and password file <none>
I20260808 02:13:46.832006 30432 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:46.832069 30432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:46.832242 30432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:46.834365 30432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "bd1d0d6ac246430683fdbae1b31dd1d0"
format_stamp: "Formatted at 2026-08-08 02:13:46 on dist-test-slave-9gcw"
server_key: "1018173ea6593efa47c58434995b214d"
server_key_iv: "58207c41487644017cc14cf1e9f4135c"
server_key_version: "encryptionkey@0"
I20260808 02:13:46.834898 30432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "bd1d0d6ac246430683fdbae1b31dd1d0"
format_stamp: "Formatted at 2026-08-08 02:13:46 on dist-test-slave-9gcw"
server_key: "1018173ea6593efa47c58434995b214d"
server_key_iv: "58207c41487644017cc14cf1e9f4135c"
server_key_version: "encryptionkey@0"
I20260808 02:13:46.838613 30432 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:13:46.840957 30449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:46.842332 30432 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:46.842464 30432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "bd1d0d6ac246430683fdbae1b31dd1d0"
format_stamp: "Formatted at 2026-08-08 02:13:46 on dist-test-slave-9gcw"
server_key: "1018173ea6593efa47c58434995b214d"
server_key_iv: "58207c41487644017cc14cf1e9f4135c"
server_key_version: "encryptionkey@0"
I20260808 02:13:46.842573 30432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:46.855851 30432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:46.858948 30432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:46.859158 30432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:46.859794 30432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:46.860646 30432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:46.860694 30432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:46.860766 30432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:46.860800 30432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:46.870028 30432 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:36433
I20260808 02:13:46.870050 30574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:36433 every 8 connection(s)
I20260808 02:13:46.870956 30432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:46.872661 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 30432
I20260808 02:13:46.872778 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:46.873102 19170 external_mini_cluster.cc:1428] Setting key 3a323d148c7314d06defae1eb3710b67
Aug 08 02:13:46 dist-test-slave-9gcw krb5kdc[30162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155226, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:13:46.886650 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:46.872918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:41165 (local address 127.18.184.190:36727)
0808 02:13:46.873072 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:46.873077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:46.874340 (+ 1263us) server_negotiation.cc:407] Connection header received
0808 02:13:46.875773 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:46.875780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:46.875845 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:46.875945 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:46.877129 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:46.878031 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:46.879198 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:46.879436 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:46.879552 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:46.882470 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:46.882473 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:46.882476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:46.882501 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:46.884383 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:46.884914 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:46.884918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:46.884920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:46.885016 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:46.885400 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:46.885404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:46.885406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:46.885616 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:46.885727 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:46.886337 (+ 610us) server_negotiation.cc:299] Negotiation successful
0808 02:13:46.886477 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
I20260808 02:13:46.887800 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:46.873417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:36727 (local address 127.18.184.130:41165)
0808 02:13:46.874170 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:46.874226 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:46.875553 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:46.876100 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:46.876106 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:46.876498 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:46.876977 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:46.876990 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:46.878169 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:46.878172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:46.879053 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:46.879063 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:46.879598 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:46.879603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:46.879778 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:46.880429 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:46.880445 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:46.882276 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:46.884507 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:46.884514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:46.884530 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:46.884803 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:46.885100 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:46.885104 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:46.885106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:46.885294 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:46.885726 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:46.885733 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:46.886000 (+ 267us) client_negotiation.cc:769] Sending connection context
0808 02:13:46.886275 (+ 275us) client_negotiation.cc:240] Negotiation successful
0808 02:13:46.886645 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":150,"threads_started":1}
I20260808 02:13:46.889030 30575 heartbeater.cc:347] Connected to a master server at 127.18.184.190:36727
I20260808 02:13:46.889377 30575 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:46.889854 30575 heartbeater.cc:511] Master 127.18.184.190:36727 requested a full tablet report, sending...
I20260808 02:13:46.891009 30211 ts_manager.cc:194] Registered new tserver with Master: bd1d0d6ac246430683fdbae1b31dd1d0 (127.18.184.130:36433)
I20260808 02:13:46.891947 30211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:41165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:13:46.904785 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36727
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:47.024020 30581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:47.024300 30581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:47.024346 30581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:47.028295 30581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:47.028357 30581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:47.028415 30581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:13:47.033862 30581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:13:47.034996 30581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:47.036057 30581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:47.046272 30586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:47.046562 30589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:47.046598 30587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:47.046773 30581 server_base.cc:1034] running on GCE node
I20260808 02:13:47.047282 30581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:47.047978 30581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:47.049207 30581 hybrid_clock.cc:648] HybridClock initialized: now 1786155227049190 us; error 35 us; skew 500 ppm
Aug 08 02:13:47 dist-test-slave-9gcw krb5kdc[30162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155227, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:47.052219 30581 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:13:47.053210 30581 webserver.cc:460] Webserver started at http://127.18.184.131:40419/ using document root <none> and password file <none>
I20260808 02:13:47.053800 30581 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:47.053901 30581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:47.054175 30581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:47.056319 30581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-9gcw"
server_key: "147b7f8f0fd02869624b5fb0cf901b10"
server_key_iv: "692c7b2b7b4da5355f44775c1b5acb59"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.056897 30581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-9gcw"
server_key: "147b7f8f0fd02869624b5fb0cf901b10"
server_key_iv: "692c7b2b7b4da5355f44775c1b5acb59"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.060722 30581 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:13:47.063408 30596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.064846 30581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:13:47.065032 30581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-9gcw"
server_key: "147b7f8f0fd02869624b5fb0cf901b10"
server_key_iv: "692c7b2b7b4da5355f44775c1b5acb59"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.065210 30581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:47.081655 30581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:47.085589 30581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:47.085778 30581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:47.086397 30581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:47.087239 30581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:47.087289 30581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.087332 30581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:47.087347 30581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.096740 30581 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:36301
I20260808 02:13:47.096820 30720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:36301 every 8 connection(s)
I20260808 02:13:47.097766 30581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:47.102677 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 30581
I20260808 02:13:47.102794 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:47.103186 19170 external_mini_cluster.cc:1428] Setting key 3e5155a525fa02434861759ae5ba313a
Aug 08 02:13:47 dist-test-slave-9gcw krb5kdc[30162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155227, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:13:47.113749 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.099874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:42173 (local address 127.18.184.190:36727)
0808 02:13:47.100039 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.100042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.100867 (+ 825us) server_negotiation.cc:407] Connection header received
0808 02:13:47.102307 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.102319 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.102381 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.102499 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:47.103959 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.104657 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.106159 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.106428 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.106560 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.109480 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:47.109486 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:47.109497 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:47.109524 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:47.111301 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.111909 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.111912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.111914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.112002 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.112420 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.112423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.112425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.112641 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:47.112767 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.113406 (+ 639us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.113545 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:13:47.114658 30726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.100222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:36727 (local address 127.18.184.131:42173)
0808 02:13:47.100684 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.100727 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:47.102088 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:47.102675 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:47.102682 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:47.103157 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:47.103805 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.103830 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.104780 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.104784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.106014 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.106025 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.106602 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.106606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.106796 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.107488 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:47.107505 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:47.109288 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:47.111443 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:47.111450 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:47.111467 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:47.111777 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:47.112118 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:47.112121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:47.112123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:47.112302 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:47.112784 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:47.112793 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:47.113068 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 02:13:47.113341 (+ 273us) client_negotiation.cc:240] Negotiation successful
0808 02:13:47.113715 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":117,"threads_started":1}
I20260808 02:13:47.116014 30723 heartbeater.cc:347] Connected to a master server at 127.18.184.190:36727
I20260808 02:13:47.116346 30723 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:47.117002 30723 heartbeater.cc:511] Master 127.18.184.190:36727 requested a full tablet report, sending...
I20260808 02:13:47.118273 30211 ts_manager.cc:194] Registered new tserver with Master: c2fe399b3f644cf7ba3ae8cac60d6c5e (127.18.184.131:36301)
I20260808 02:13:47.119029 30211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:42173
I20260808 02:13:47.129870 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:13:47.140722 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.132108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39064 (local address 127.18.184.190:36727)
0808 02:13:47.132246 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.132251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.132500 (+ 249us) server_negotiation.cc:407] Connection header received
0808 02:13:47.132707 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.132710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.132765 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.132839 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:47.133792 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.134363 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.135279 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.135541 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.135628 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.137231 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:47.137236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:47.137242 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:47.137275 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:47.138986 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.139493 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.139501 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.139502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.139586 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.139921 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.139923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.139924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.140164 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:47.140272 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.140491 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.140594 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:13:47.144034 30211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:47.147660 30211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:47.163407 30735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.156863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:36301 (local address 127.0.0.1:34858)
0808 02:13:47.157286 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.157351 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:47.157492 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:47.158837 (+ 1345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:47.158843 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:47.158866 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:47.159025 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.159032 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.160493 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.160496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.161732 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.161742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.162850 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.162853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.162965 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.163007 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 02:13:47.163121 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 02:13:47.163240 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":38,"thread_start_us":113,"threads_started":1}
I20260808 02:13:47.163476 30736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.157031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34858 (local address 127.18.184.131:36301)
0808 02:13:47.158370 (+ 1339us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.158384 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.158410 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:13:47.158486 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.158494 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.158675 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.158852 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:47.159141 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.160353 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.161869 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.162707 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.162843 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.162999 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.163179 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.163292 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1230,"thread_start_us":164,"threads_started":1}
I20260808 02:13:47.164086 30740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.157888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40864 (local address 127.18.184.129:41457)
0808 02:13:47.158301 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.158317 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.158446 (+ 129us) server_negotiation.cc:407] Connection header received
0808 02:13:47.158566 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.158570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.158723 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.158856 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:47.159352 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.160483 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.162082 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.162854 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.162963 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.163082 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.163829 (+ 747us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.163939 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":98,"threads_started":1}
I20260808 02:13:47.164412 30737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.157655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:41457 (local address 127.0.0.1:40864)
0808 02:13:47.158330 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.158345 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:47.158451 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:47.159019 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:47.159022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:47.159053 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:47.159216 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.159224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.160630 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.160633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.161960 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.161969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.163172 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.163176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.163278 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.163302 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:13:47.163377 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 02:13:47.163447 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":48,"spinlock_wait_cycles":5248,"thread_start_us":105,"threads_started":1}
I20260808 02:13:47.164769 30738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.157309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:36433 (local address 127.0.0.1:47454)
0808 02:13:47.157962 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.157978 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:47.158073 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:47.159019 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:47.159022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:47.159054 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:47.159207 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.159224 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.161884 (+ 2660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.161887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.162666 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.162674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.164358 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.164362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.164455 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.164479 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:13:47.164545 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 02:13:47.164615 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":56,"spinlock_wait_cycles":3840,"thread_start_us":105,"threads_started":1}
I20260808 02:13:47.165423 30739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.157773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47454 (local address 127.18.184.130:36433)
0808 02:13:47.158317 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.158330 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.158357 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:13:47.158461 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.158469 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.158686 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.158852 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:47.160538 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.161726 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.162809 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.164002 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.164120 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.165117 (+ 997us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.165188 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.165280 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":147,"threads_started":1}
I20260808 02:13:47.166920 30649 tablet_service.cc:1467] Processing CreateTablet for tablet 7772e6f9d2714660bbd9a28be821de52 (DEFAULT_TABLE table=test-table [id=3ebf9f87d27a44bdbec1d9c7a9733b0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:47.167152 30352 tablet_service.cc:1467] Processing CreateTablet for tablet 7772e6f9d2714660bbd9a28be821de52 (DEFAULT_TABLE table=test-table [id=3ebf9f87d27a44bdbec1d9c7a9733b0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:47.168054 30649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7772e6f9d2714660bbd9a28be821de52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:47.168205 30352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7772e6f9d2714660bbd9a28be821de52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:47.167900 30503 tablet_service.cc:1467] Processing CreateTablet for tablet 7772e6f9d2714660bbd9a28be821de52 (DEFAULT_TABLE table=test-table [id=3ebf9f87d27a44bdbec1d9c7a9733b0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:47.168911 30503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7772e6f9d2714660bbd9a28be821de52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:47.174698 30741 tablet_bootstrap.cc:492] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e: Bootstrap starting.
I20260808 02:13:47.175568 30742 tablet_bootstrap.cc:492] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9: Bootstrap starting.
I20260808 02:13:47.177230 30741 tablet_bootstrap.cc:654] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:47.177497 30743 tablet_bootstrap.cc:492] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0: Bootstrap starting.
I20260808 02:13:47.178665 30741 log.cc:824] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:47.179704 30742 tablet_bootstrap.cc:654] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:47.180171 30743 tablet_bootstrap.cc:654] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:47.180606 30742 log.cc:824] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:47.180644 30741 tablet_bootstrap.cc:492] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e: No bootstrap required, opened a new log
I20260808 02:13:47.180860 30741 ts_tablet_manager.cc:1397] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260808 02:13:47.180948 30743 log.cc:824] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:47.182555 30742 tablet_bootstrap.cc:492] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9: No bootstrap required, opened a new log
I20260808 02:13:47.182747 30742 ts_tablet_manager.cc:1397] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 02:13:47.182936 30743 tablet_bootstrap.cc:492] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0: No bootstrap required, opened a new log
I20260808 02:13:47.183122 30743 ts_tablet_manager.cc:1397] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:13:47.184491 30741 raft_consensus.cc:348] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36301 } } peers { permanent_uuid: "a7f01e4714bd426e98652c00312b23d9" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 41457 } } peers { permanent_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 36433 } }
I20260808 02:13:47.184753 30741 raft_consensus.cc:374] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:47.184806 30741 raft_consensus.cc:729] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2fe399b3f644cf7ba3ae8cac60d6c5e, State: Initialized, Role: FOLLOWER
I20260808 02:13:47.185314 30741 consensus_queue.cc:260] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36301 } } peers { permanent_uuid: "a7f01e4714bd426e98652c00312b23d9" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 41457 } } peers { permanent_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 36433 } }
I20260808 02:13:47.186280 30741 ts_tablet_manager.cc:1428] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 02:13:47.186663 30723 heartbeater.cc:503] Master 127.18.184.190:36727 was elected leader, sending a full tablet report...
I20260808 02:13:47.187104 30743 raft_consensus.cc:348] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36301 } } peers { permanent_uuid: "a7f01e4714bd426e98652c00312b23d9" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 41457 } } peers { permanent_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 36433 } }
I20260808 02:13:47.187376 30743 raft_consensus.cc:374] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:47.187232 30742 raft_consensus.cc:348] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36301 } } peers { permanent_uuid: "a7f01e4714bd426e98652c00312b23d9" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 41457 } } peers { permanent_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 36433 } }
I20260808 02:13:47.187440 30743 raft_consensus.cc:729] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd1d0d6ac246430683fdbae1b31dd1d0, State: Initialized, Role: FOLLOWER
I20260808 02:13:47.187506 30742 raft_consensus.cc:374] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:47.187559 30742 raft_consensus.cc:729] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7f01e4714bd426e98652c00312b23d9, State: Initialized, Role: FOLLOWER
I20260808 02:13:47.188000 30742 consensus_queue.cc:260] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36301 } } peers { permanent_uuid: "a7f01e4714bd426e98652c00312b23d9" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 41457 } } peers { permanent_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 36433 } }
I20260808 02:13:47.188000 30743 consensus_queue.cc:260] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36301 } } peers { permanent_uuid: "a7f01e4714bd426e98652c00312b23d9" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 41457 } } peers { permanent_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 36433 } }
I20260808 02:13:47.188953 30575 heartbeater.cc:503] Master 127.18.184.190:36727 was elected leader, sending a full tablet report...
I20260808 02:13:47.188992 30742 ts_tablet_manager.cc:1428] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:13:47.188992 30743 ts_tablet_manager.cc:1428] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:13:47.189395 30426 heartbeater.cc:503] Master 127.18.184.190:36727 was elected leader, sending a full tablet report...
I20260808 02:13:47.244441 30748 raft_consensus.cc:482] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:47.244668 30748 raft_consensus.cc:504] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36301 } } peers { permanent_uuid: "a7f01e4714bd426e98652c00312b23d9" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 41457 } } peers { permanent_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 36433 } }
I20260808 02:13:47.246083 30748 leader_election.cc:288] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2fe399b3f644cf7ba3ae8cac60d6c5e (127.18.184.131:36301), a7f01e4714bd426e98652c00312b23d9 (127.18.184.129:41457)
I20260808 02:13:47.250280 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.246336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:36301 (local address 127.18.184.130:34499)
0808 02:13:47.246455 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.246471 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:47.246579 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:47.246966 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:47.246970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:47.246993 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:47.247123 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.247132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.248069 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.248072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.248985 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.248995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.249771 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.249775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.249934 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.249967 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:13:47.250040 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 02:13:47.250118 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":21}
I20260808 02:13:47.250341 30736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.246494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:34499 (local address 127.18.184.131:36301)
0808 02:13:47.246647 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.246654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.246674 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:13:47.246738 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.246740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.246797 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.246916 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:47.247268 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.247953 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.249124 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.249657 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.249753 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.249798 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.250077 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.250154 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:13:47.250451 30740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.246828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:57501 (local address 127.18.184.129:41457)
0808 02:13:47.246967 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.246971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.247145 (+ 174us) server_negotiation.cc:407] Connection header received
0808 02:13:47.247255 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.247260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.247306 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.247367 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:47.247654 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.248231 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.249408 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.249865 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.249985 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.250049 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.250249 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.250335 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:13:47.250888 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.246619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:41457 (local address 127.18.184.130:57501)
0808 02:13:47.247042 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.247056 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:47.247157 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:47.247410 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:47.247412 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:47.247431 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:47.247546 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.247552 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.248358 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.248361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.249287 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.249299 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.249990 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.249993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.250117 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.250141 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:13:47.250221 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 02:13:47.250285 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":1,"thread_start_us":165,"threads_started":1}
I20260808 02:13:47.251055 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7772e6f9d2714660bbd9a28be821de52" candidate_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e" is_pre_election: true
I20260808 02:13:47.251452 30670 raft_consensus.cc:2454] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd1d0d6ac246430683fdbae1b31dd1d0 in term 0.
I20260808 02:13:47.251541 30372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7772e6f9d2714660bbd9a28be821de52" candidate_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7f01e4714bd426e98652c00312b23d9" is_pre_election: true
I20260808 02:13:47.251823 30372 raft_consensus.cc:2454] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd1d0d6ac246430683fdbae1b31dd1d0 in term 0.
I20260808 02:13:47.251945 30452 leader_election.cc:302] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd1d0d6ac246430683fdbae1b31dd1d0, c2fe399b3f644cf7ba3ae8cac60d6c5e; no voters:
I20260808 02:13:47.252215 30748 raft_consensus.cc:2781] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:47.252308 30748 raft_consensus.cc:482] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:47.252358 30748 raft_consensus.cc:3037] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:47.253777 30748 raft_consensus.cc:504] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36301 } } peers { permanent_uuid: "a7f01e4714bd426e98652c00312b23d9" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 41457 } } peers { permanent_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 36433 } }
I20260808 02:13:47.254161 30748 leader_election.cc:288] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [CANDIDATE]: Term 1 election: Requested vote from peers c2fe399b3f644cf7ba3ae8cac60d6c5e (127.18.184.131:36301), a7f01e4714bd426e98652c00312b23d9 (127.18.184.129:41457)
I20260808 02:13:47.254551 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7772e6f9d2714660bbd9a28be821de52" candidate_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e"
I20260808 02:13:47.254619 30372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7772e6f9d2714660bbd9a28be821de52" candidate_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7f01e4714bd426e98652c00312b23d9"
I20260808 02:13:47.254683 30670 raft_consensus.cc:3037] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:47.254813 30372 raft_consensus.cc:3037] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:47.255843 30670 raft_consensus.cc:2454] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd1d0d6ac246430683fdbae1b31dd1d0 in term 1.
I20260808 02:13:47.256040 30372 raft_consensus.cc:2454] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd1d0d6ac246430683fdbae1b31dd1d0 in term 1.
I20260808 02:13:47.256206 30452 leader_election.cc:302] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd1d0d6ac246430683fdbae1b31dd1d0, c2fe399b3f644cf7ba3ae8cac60d6c5e; no voters:
I20260808 02:13:47.256398 30748 raft_consensus.cc:2781] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:47.256667 30748 raft_consensus.cc:686] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 1 LEADER]: Becoming Leader. State: Replica: bd1d0d6ac246430683fdbae1b31dd1d0, State: Running, Role: LEADER
I20260808 02:13:47.257057 30748 consensus_queue.cc:237] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36301 } } peers { permanent_uuid: "a7f01e4714bd426e98652c00312b23d9" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 41457 } } peers { permanent_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 36433 } }
I20260808 02:13:47.260378 30210 catalog_manager.cc:5495] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd1d0d6ac246430683fdbae1b31dd1d0 (127.18.184.130). New cstate: current_term: 1 leader_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7f01e4714bd426e98652c00312b23d9" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 41457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd1d0d6ac246430683fdbae1b31dd1d0" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 36433 } health_report { overall_health: HEALTHY } } }
I20260808 02:13:47.286940 30739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.283122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47464 (local address 127.18.184.130:36433)
0808 02:13:47.283353 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.283358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.283376 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:13:47.283465 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.283468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.283527 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.283643 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:47.283931 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.284548 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.285447 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.285661 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.285766 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.286107 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:47.286529 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:47.286668 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.286710 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.286767 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:13:47.292824 30670 raft_consensus.cc:1261] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e [term 1 FOLLOWER]: Refusing update from remote peer bd1d0d6ac246430683fdbae1b31dd1d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:47.292872 30372 raft_consensus.cc:1261] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9 [term 1 FOLLOWER]: Refusing update from remote peer bd1d0d6ac246430683fdbae1b31dd1d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:47.293619 30751 consensus_queue.cc:1035] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2fe399b3f644cf7ba3ae8cac60d6c5e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:47.293840 30748 consensus_queue.cc:1035] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7f01e4714bd426e98652c00312b23d9" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 41457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:47.303507 30759 mvcc.cc:205] Tried to move back new op lower bound from 7316091810983919616 to 7316091810847973376. Current Snapshot: MvccSnapshot[applied={T|T < 7316091810983919616}]
I20260808 02:13:47.304175 30757 mvcc.cc:205] Tried to move back new op lower bound from 7316091810983919616 to 7316091810847973376. Current Snapshot: MvccSnapshot[applied={T|T < 7316091810983919616}]
I20260808 02:13:47.310954 30209 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:47.311139 30209 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:47.313604 30209 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b88f9b51a5c4a418cf3bbc5d51a87b3: Sending DeleteTablet for 3 replicas of tablet 7772e6f9d2714660bbd9a28be821de52
I20260808 02:13:47.314455 30503 tablet_service.cc:1514] Processing DeleteTablet for tablet 7772e6f9d2714660bbd9a28be821de52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:47 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:47454
I20260808 02:13:47.314456 30649 tablet_service.cc:1514] Processing DeleteTablet for tablet 7772e6f9d2714660bbd9a28be821de52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:47 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:34858
I20260808 02:13:47.314456 30352 tablet_service.cc:1514] Processing DeleteTablet for tablet 7772e6f9d2714660bbd9a28be821de52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:47 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:40864
I20260808 02:13:47.315251 30767 tablet_replica.cc:330] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9: stopping tablet replica
I20260808 02:13:47.315598 30767 raft_consensus.cc:2229] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:47.315675 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 30278
I20260808 02:13:47.315918 30767 raft_consensus.cc:2258] T 7772e6f9d2714660bbd9a28be821de52 P a7f01e4714bd426e98652c00312b23d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:47.317166 30769 tablet_replica.cc:330] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e: stopping tablet replica
I20260808 02:13:47.317526 30769 raft_consensus.cc:2229] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:47.317849 30769 raft_consensus.cc:2258] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:47.319598 30769 ts_tablet_manager.cc:1905] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:47.321394 30768 tablet_replica.cc:330] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0: stopping tablet replica
I20260808 02:13:47.321688 30768 raft_consensus.cc:2229] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:47.322024 30768 raft_consensus.cc:2258] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:47.322853 30769 ts_tablet_manager.cc:1918] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:47.322956 30769 log.cc:1196] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/7772e6f9d2714660bbd9a28be821de52
I20260808 02:13:47.323297 30769 ts_tablet_manager.cc:1939] T 7772e6f9d2714660bbd9a28be821de52 P c2fe399b3f644cf7ba3ae8cac60d6c5e: Deleting consensus metadata
I20260808 02:13:47.323316 30768 ts_tablet_manager.cc:1905] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:47.325234 30196 catalog_manager.cc:4841] TS c2fe399b3f644cf7ba3ae8cac60d6c5e (127.18.184.131:36301): tablet 7772e6f9d2714660bbd9a28be821de52 (table test-table [id=3ebf9f87d27a44bdbec1d9c7a9733b0d]) successfully deleted
I20260808 02:13:47.325982 30768 ts_tablet_manager.cc:1918] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:47.326081 30768 log.cc:1196] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/7772e6f9d2714660bbd9a28be821de52
I20260808 02:13:47.326427 30768 ts_tablet_manager.cc:1939] T 7772e6f9d2714660bbd9a28be821de52 P bd1d0d6ac246430683fdbae1b31dd1d0: Deleting consensus metadata
W20260808 02:13:47.327533 30196 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:41457 (error 108)
I20260808 02:13:47.327533 30195 catalog_manager.cc:4841] TS bd1d0d6ac246430683fdbae1b31dd1d0 (127.18.184.130:36433): tablet 7772e6f9d2714660bbd9a28be821de52 (table test-table [id=3ebf9f87d27a44bdbec1d9c7a9733b0d]) successfully deleted
I20260808 02:13:47.328470 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 30432
I20260808 02:13:47.328824 30737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.328376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:41457 (local address 127.0.0.1:40866)
0808 02:13:47.328549 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.328612 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:41457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260808 02:13:47.331406 30196 catalog_manager.cc:4584] TS a7f01e4714bd426e98652c00312b23d9 (127.18.184.129:41457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7772e6f9d2714660bbd9a28be821de52: Network error: Client connection negotiation failed: client connection to 127.18.184.129:41457: connect: Connection refused (error 111)
I20260808 02:13:47.336531 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 30581
I20260808 02:13:47.343513 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 30175
2026-08-08T02:13:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2460 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:47 dist-test-slave-9gcw krb5kdc[30773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:47 dist-test-slave-9gcw krb5kdc[30773](info): set up 2 sockets
Aug 08 02:13:47 dist-test-slave-9gcw krb5kdc[30773](info): commencing operation
krb5kdc: starting...
W20260808 02:13:48.453476 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:13:48 dist-test-slave-9gcw krb5kdc[30773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.184.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.184.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/rangeradmin_127.18.184.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/rangeradmin_127.18.184.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.184.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.184.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/rangerlookup_127.18.184.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/rangerlookup_127.18.184.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.184.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.184.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/HTTP_127.18.184.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/HTTP_127.18.184.148@KRBTEST.COM.keytab.
I20260808 02:13:48.642699 19170 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmhslVu/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/postgres -l logfile start
2026-08-08 02:13:51.487 UTC [30804] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 02:13:51.487 UTC [30804] LOG: listening on IPv4 address "127.18.184.148", port 56627
2026-08-08 02:13:51.491 UTC [30804] LOG: listening on Unix socket "/tmp/.s.PGSQL.56627"
2026-08-08 02:13:51.515 UTC [30807] LOG: database system was shut down at 2026-08-08 02:13:49 UTC
2026-08-08 02:13:51.519 UTC [30804] LOG: database system is ready to accept connections
I20260808 02:13:52.508524 19170 mini_postgres.cc:96] Postgres bound to 56627
2026-08-08 02:13:52.518 UTC [30816] FATAL: database "slave" does not exist
127.18.184.148:56627 - accepting connections
I20260808 02:13:52.519243 19170 mini_ranger.cc:161] Starting Ranger...
I20260808 02:13:52.536757 19170 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 02:13:53.098583 19170 mini_ranger.cc:87] Created ranger Postgres database
I20260808 02:13:53.098703 19170 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-admin
2026-08-08 02:13:53,431 [I] DB FLAVOR :POSTGRES
2026-08-08 02:13:53,431 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:13:53,431 [I] Checking connection..
2026-08-08 02:13:53,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 02:13:53,712 [I] Checking connection passed.
2026-08-08 02:13:53,712 [I] --------- Verifying version history table ---------
2026-08-08 02:13:53,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:13:53,985 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:13:53,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:13:54,255 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:13:54,255 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 02:13:54,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 02:13:54.528 UTC [30885] WARNING: there is no transaction in progress
2026-08-08 02:13:54,537 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 02:13:54,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:13:54,802 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 02:13:54,802 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:13:54,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 02:13:55,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 02:13:55,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9gcw.c.gcp-upstream.internal','N') ;"
2026-08-08 02:13:55,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:13:55,883 [I] Table x_portal_user does not exist in database ranger
2026-08-08 02:13:55,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:13:56,139 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 02:13:56,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:13:56,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:13:56,675 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 02:13:56,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 02:13:57.746 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.765 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.784 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.802 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.820 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.842 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.909 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.918 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.936 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.948 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.961 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.977 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.989 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:57.997 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:58.007 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:58.015 UTC [31001] WARNING: there is no transaction in progress
2026-08-08 02:13:58,791 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 02:13:58,791 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 02:13:59,043 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 02:13:59,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:13:59,312 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 02:13:59,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:13:59,570 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 02:13:59,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:13:59,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:14:00,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9gcw.c.gcp-upstream.internal';"
2026-08-08 02:14:00,403 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 02:14:00,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 02:14:00,679 [I] DB_PATCHES have already been applied
I20260808 02:14:00.685895 19170 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmhslVu/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 02:14:00.685968 19170 mini_ranger.cc:193] Using host: 127.18.184.148
I20260808 02:14:00.691010 19170 mini_ranger.cc:227] Ranger admin URL: http://127.18.184.148:48603
Aug 08, 2026 2:14:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:14:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:14:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:14:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:14:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:14:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:14:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 02:14:01.938057 19170 mini_ranger.cc:160] Time spent starting Ranger: real 9.419s user 0.000s sys 0.014s
Aug 08, 2026 2:14:22 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:14:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 2:14:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:14:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:14:22 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:14:23 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 02:14:24.235133 19170 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.184.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.184.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/rangerkms_127.18.184.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/rangerkms_127.18.184.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.184.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.184.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.184.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/HTTP_127.18.184.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.184.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/HTTP_127.18.184.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 02:14:24 dist-test-slave-9gcw krb5kdc[30773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155264, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 02:14:24.340282 19170 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 02:14:24.352576 19170 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 02:14:25.335414 19170 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 02:14:25.335548 19170 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-kms
I20260808 02:14:25.335667 19170 mini_ranger_kms.cc:227] Using postgres at 127.18.184.148:56627
2026-08-08 02:14:25,646 [I] DB FLAVOR :POSTGRES
2026-08-08 02:14:25,646 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:14:25,647 [I] Checking connection
2026-08-08 02:14:25,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 02:14:25,924 [I] connection success
2026-08-08 02:14:25,924 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 02:14:25,924 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 02:14:25,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 02:14:26,202 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 02:14:26,203 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:14:26,203 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 02:14:26,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:56627/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 02:14:26,488 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 02:14:26.972035 19170 mini_ranger_kms.cc:310] Created kms service
I20260808 02:14:27.159444 19170 mini_ranger_kms.cc:326] Created kudu user
I20260808 02:14:27.210685 19170 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 02:14:27 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155264, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:14:27.679217 19170 mini_ranger_kms.cc:384] Added ranger policy
I20260808 02:14:27.679396 19170 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 02:14:27.679431 19170 mini_ranger_kms.cc:240] Using host: 127.18.184.148
I20260808 02:14:27.684959 19170 mini_ranger_kms.cc:276] Ranger KMS PID: 31195
I20260808 02:14:27.685129 19170 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.184.148:50335
Aug 08, 2026 2:14:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:14:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:14:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:14:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:14:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:14:29 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 02:14:30.147966 19170 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.808s user 0.000s sys 0.287s
I20260808 02:14:30.148123 19170 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 02:14:30.148249 19170 mini_ranger_kms.cc:401] 127.18.184.148:50335/kms/v1/keys
Aug 08 02:14:30 dist-test-slave-9gcw krb5kdc[30773](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155270, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.184.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 02:14:30 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155270, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.184.148@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
Aug 08 02:14:31 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155270, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.184.148@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
Aug 08 02:14:33 dist-test-slave-9gcw krb5kdc[30773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:14:33.476241 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:46109
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:46109
--ranger_config_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.184.148:50335/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:14:33.606611 31251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:14:33.606926 31251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:14:33.607002 31251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:14:33.611671 31251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:14:33.611755 31251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:14:33.611778 31251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:14:33.611795 31251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:14:33.611850 31251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:14:33.618363 31251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.184.148:50335/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:46109
--ranger_config_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:46109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:14:33.619608 31251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:14:33.620788 31251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:14:33.631826 31257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:33.631858 31258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:14:33.632021 31251 server_base.cc:1034] running on GCE node
W20260808 02:14:33.631827 31260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:14:33.632910 31251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:14:33.634238 31251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:14:33.635421 31251 hybrid_clock.cc:648] HybridClock initialized: now 1786155273635420 us; error 53 us; skew 500 ppm
Aug 08 02:14:33 dist-test-slave-9gcw krb5kdc[30773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155273, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:14:33.644546 31251 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:14:33.645871 31251 webserver.cc:460] Webserver started at http://127.18.184.190:44897/ using document root <none> and password file <none>
I20260808 02:14:33.646483 31251 fs_manager.cc:359] Metadata directory not provided
I20260808 02:14:33.646548 31251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:14:33.646796 31251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:14:33 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155273, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:14:33.711894 31251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "051c89e71f204680aefc6608d7f750b5"
format_stamp: "Formatted at 2026-08-08 02:14:33 on dist-test-slave-9gcw"
server_key: "a5ff3fdd0df2f6d84c45900a74acb458"
server_key_iv: "3613aa685fac6ae9b6e32601cb2cb4ee"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:33.712682 31251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "051c89e71f204680aefc6608d7f750b5"
format_stamp: "Formatted at 2026-08-08 02:14:33 on dist-test-slave-9gcw"
server_key: "a5ff3fdd0df2f6d84c45900a74acb458"
server_key_iv: "3613aa685fac6ae9b6e32601cb2cb4ee"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:33.718214 31251 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 02:14:33.736469 31267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:33.738158 31251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:14:33.738322 31251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "051c89e71f204680aefc6608d7f750b5"
format_stamp: "Formatted at 2026-08-08 02:14:33 on dist-test-slave-9gcw"
server_key: "a5ff3fdd0df2f6d84c45900a74acb458"
server_key_iv: "3613aa685fac6ae9b6e32601cb2cb4ee"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:33.738469 31251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:14:33.766220 31251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:14:33.770044 31251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:14:33.770277 31251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:14:33.778277 31251 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:46109
I20260808 02:14:33.778316 31324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:46109 every 8 connection(s)
I20260808 02:14:33.779342 31251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:14:33.782837 31325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:14:33.786881 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 31251
I20260808 02:14:33.787034 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:14:33.787367 19170 external_mini_cluster.cc:1428] Setting key 8fd515f727d8dcf2666fba205e869e72
I20260808 02:14:33.790010 31325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5: Bootstrap starting.
I20260808 02:14:33.792789 31325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5: Neither blocks nor log segments found. Creating new log.
I20260808 02:14:33.793705 31325 log.cc:824] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5: Log is configured to *not* fsync() on all Append() calls
I20260808 02:14:33.796089 31325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5: No bootstrap required, opened a new log
Aug 08 02:14:33 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:14:33.800733 31325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051c89e71f204680aefc6608d7f750b5" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46109 } }
I20260808 02:14:33.801033 31325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:14:33.801121 31325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c89e71f204680aefc6608d7f750b5, State: Initialized, Role: FOLLOWER
I20260808 02:14:33.801723 31325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051c89e71f204680aefc6608d7f750b5" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46109 } }
I20260808 02:14:33.801996 31325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:14:33.802114 31325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:14:33.802258 31325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:14:33.802980 31328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:33.788917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60322 (local address 127.18.184.190:46109)
0808 02:14:33.789427 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:33.789446 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:33.789485 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:14:33.790210 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:33.790230 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:33.790567 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:33.790921 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:14:33.791824 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:33.792860 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:33.793983 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:33.794699 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:33.794889 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:33.797813 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:14:33.797819 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:14:33.797833 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:14:33.797864 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:14:33.799668 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:33.800182 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:33.800189 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:33.800194 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:33.800314 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:33.800665 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:33.800670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:33.800673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:33.801218 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:14:33.801574 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:33.801806 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 02:14:33.802135 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":199,"threads_started":1}
I20260808 02:14:33.803462 31325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051c89e71f204680aefc6608d7f750b5" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46109 } }
I20260808 02:14:33.803851 31325 leader_election.cc:302] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 051c89e71f204680aefc6608d7f750b5; no voters:
I20260808 02:14:33.804234 31325 leader_election.cc:288] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:14:33.804473 31331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:14:33.804750 31331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [term 1 LEADER]: Becoming Leader. State: Replica: 051c89e71f204680aefc6608d7f750b5, State: Running, Role: LEADER
I20260808 02:14:33.805186 31331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051c89e71f204680aefc6608d7f750b5" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46109 } }
I20260808 02:14:33.805672 31325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:14:33.806602 31331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 051c89e71f204680aefc6608d7f750b5. Latest consensus state: current_term: 1 leader_uuid: "051c89e71f204680aefc6608d7f750b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051c89e71f204680aefc6608d7f750b5" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46109 } } }
I20260808 02:14:33.806814 31331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [sys.catalog]: This master's current role is: LEADER
I20260808 02:14:33.809206 31332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "051c89e71f204680aefc6608d7f750b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051c89e71f204680aefc6608d7f750b5" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46109 } } }
I20260808 02:14:33.809279 31339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:14:33.809334 31332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5 [sys.catalog]: This master's current role is: LEADER
I20260808 02:14:33.809864 31325 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 02:14:33.812413 31339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:14:33.818864 31339 catalog_manager.cc:1347] Generated new cluster ID: 31c4aea1ce6642619c463ecc636d7da2
I20260808 02:14:33.818941 31339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:14:33.835644 31339 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:14:33.837285 31339 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:14:33.848837 31339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5: Generated new TSK 0
I20260808 02:14:33.849895 31339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:14:34.851496 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.184.148:50335/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46109
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:14:35.062341 31372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:14:35.062660 31372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:14:35.062711 31372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:14:35.069525 31372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:14:35.069717 31372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:14:35.070056 31372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:14:35.080899 31372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.184.148:50335/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:14:35.084226 31372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:14:35.086585 31372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:14:35.105641 31379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:35.105696 31378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:14:35.107074 31372 server_base.cc:1034] running on GCE node
W20260808 02:14:35.107275 31381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:14:35.108246 31372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:14:35.110424 31372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:14:35.117180 31372 hybrid_clock.cc:648] HybridClock initialized: now 1786155275117053 us; error 84 us; skew 500 ppm
Aug 08 02:14:35 dist-test-slave-9gcw krb5kdc[30773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:14:35.125907 31372 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:14:35.129247 31372 webserver.cc:460] Webserver started at http://127.18.184.129:43199/ using document root <none> and password file <none>
I20260808 02:14:35.130743 31372 fs_manager.cc:359] Metadata directory not provided
I20260808 02:14:35.131089 31372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:14:35.131414 31372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:14:35 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:14:35.166469 31372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "5d8058ffc81845e3aea8b9f29bd833dd"
format_stamp: "Formatted at 2026-08-08 02:14:35 on dist-test-slave-9gcw"
server_key: "891921ed6973a1421c6ef51707fc9bda"
server_key_iv: "9c2d78d85f54e65405c9499095affb47"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:35.169159 31372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "5d8058ffc81845e3aea8b9f29bd833dd"
format_stamp: "Formatted at 2026-08-08 02:14:35 on dist-test-slave-9gcw"
server_key: "891921ed6973a1421c6ef51707fc9bda"
server_key_iv: "9c2d78d85f54e65405c9499095affb47"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:35.187508 31372 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.007s sys 0.003s
I20260808 02:14:35.252437 31390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:35.259583 31372 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.006s sys 0.000s
I20260808 02:14:35.260145 31372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "5d8058ffc81845e3aea8b9f29bd833dd"
format_stamp: "Formatted at 2026-08-08 02:14:35 on dist-test-slave-9gcw"
server_key: "891921ed6973a1421c6ef51707fc9bda"
server_key_iv: "9c2d78d85f54e65405c9499095affb47"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:35.261106 31372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:14:35.334826 31372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:14:35.346665 31372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:14:35.347470 31372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:14:35.349865 31372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:14:35.352523 31372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:14:35.352761 31372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 02:14:35.352926 31372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:14:35.353027 31372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:35.378911 31372 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:36497
I20260808 02:14:35.380877 31372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:14:35.386248 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 31372
I20260808 02:14:35.386395 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:14:35.387207 19170 external_mini_cluster.cc:1428] Setting key a3330bc743598b683644df3d2dd6b1f0
I20260808 02:14:35.397891 31512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:36497 every 8 connection(s)
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
Aug 08 02:14:35 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:14:35.468475 31517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:35.403028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:46109 (local address 127.18.184.129:41103)
0808 02:14:35.407209 (+ 4181us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:35.407478 (+ 269us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:35.411580 (+ 4102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:35.423038 (+ 11458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:35.423073 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:35.424572 (+ 1499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:14:35.427804 (+ 3232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:35.427934 (+ 130us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:35.443813 (+ 15879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:35.443818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:35.446252 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:35.446270 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:35.447218 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:35.447222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:35.447464 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:35.449294 (+ 1830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:14:35.449326 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:14:35.452560 (+ 3234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:14:35.460482 (+ 7922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:14:35.460513 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:14:35.460586 (+ 73us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:14:35.461533 (+ 947us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:14:35.462452 (+ 919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:14:35.462508 (+ 56us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:14:35.462520 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:14:35.463245 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:14:35.464879 (+ 1634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:14:35.464897 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:14:35.465821 (+ 924us) client_negotiation.cc:769] Sending connection context
0808 02:14:35.466276 (+ 455us) client_negotiation.cc:240] Negotiation successful
0808 02:14:35.466687 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3557,"thread_start_us":293,"threads_started":1}
I20260808 02:14:35.469930 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:35.401096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:41103 (local address 127.18.184.190:46109)
0808 02:14:35.410424 (+ 9328us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:35.410461 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:35.410576 (+ 115us) server_negotiation.cc:407] Connection header received
0808 02:14:35.415474 (+ 4898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:35.415542 (+ 68us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:35.415778 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:35.416081 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:14:35.441358 (+ 25277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:35.443362 (+ 2004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:35.446443 (+ 3081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:35.446841 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:35.446967 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:35.452899 (+ 5932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:14:35.452909 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:14:35.452935 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:14:35.453072 (+ 137us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:14:35.460042 (+ 6970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:35.461913 (+ 1871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:35.461920 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:35.461926 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:35.462151 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:35.463698 (+ 1547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:35.463702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:35.463704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:35.464301 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:14:35.464708 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:35.469170 (+ 4462us) server_negotiation.cc:299] Negotiation successful
0808 02:14:35.469365 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8970,"thread_start_us":229,"threads_started":1}
I20260808 02:14:35.471652 31513 heartbeater.cc:347] Connected to a master server at 127.18.184.190:46109
I20260808 02:14:35.471975 31513 heartbeater.cc:464] Registering TS with master...
I20260808 02:14:35.473240 31513 heartbeater.cc:511] Master 127.18.184.190:46109 requested a full tablet report, sending...
I20260808 02:14:35.475817 31286 ts_manager.cc:194] Registered new tserver with Master: 5d8058ffc81845e3aea8b9f29bd833dd (127.18.184.129:36497)
I20260808 02:14:35.477795 31286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:41103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
W20260808 02:14:35.493670 19170 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.18.184.130: real 0.102s user 0.003s sys 0.008s
I20260808 02:14:35.494823 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.184.148:50335/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46109
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:14:35 dist-test-slave-9gcw krb5kdc[30773](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 02:14:35.931195 31521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:14:35.932533 31521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:14:35.932943 31521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:14:35.947991 31521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:14:35.948326 31521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:14:35.948695 31521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:14:35.962050 31521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.184.148:50335/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:14:35.964393 31521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:14:35.966765 31521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:14:35.980968 31528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:35.981559 31530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:35.981701 31527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:14:35.982020 31521 server_base.cc:1034] running on GCE node
I20260808 02:14:35.982769 31521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:14:35.983861 31521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:14:35.987268 31521 hybrid_clock.cc:648] HybridClock initialized: now 1786155275987104 us; error 139 us; skew 500 ppm
Aug 08 02:14:35 dist-test-slave-9gcw krb5kdc[30773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:14:35.996078 31521 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:14:35.997539 31521 webserver.cc:460] Webserver started at http://127.18.184.130:34075/ using document root <none> and password file <none>
I20260808 02:14:35.999990 31521 fs_manager.cc:359] Metadata directory not provided
I20260808 02:14:36.000450 31521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:14:36.001456 31521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:14:36 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:14:36.040495 31521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "e8d59c5411784deeaf1540061d60f8e8"
format_stamp: "Formatted at 2026-08-08 02:14:36 on dist-test-slave-9gcw"
server_key: "b56eeefe76c67deb3fbd46f8aad22560"
server_key_iv: "12ec65b6de6b3cf69bf2748be773a2a4"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:36.042676 31521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "e8d59c5411784deeaf1540061d60f8e8"
format_stamp: "Formatted at 2026-08-08 02:14:36 on dist-test-slave-9gcw"
server_key: "b56eeefe76c67deb3fbd46f8aad22560"
server_key_iv: "12ec65b6de6b3cf69bf2748be773a2a4"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:36.056384 31521 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.011s sys 0.000s
I20260808 02:14:36.080487 31539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:36.084224 31521 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260808 02:14:36.084388 31521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "e8d59c5411784deeaf1540061d60f8e8"
format_stamp: "Formatted at 2026-08-08 02:14:36 on dist-test-slave-9gcw"
server_key: "b56eeefe76c67deb3fbd46f8aad22560"
server_key_iv: "12ec65b6de6b3cf69bf2748be773a2a4"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:36.084534 31521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:14:36.113294 31521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:14:36.117261 31521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:14:36.117477 31521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:14:36.121788 31521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:14:36.122939 31521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:14:36.123008 31521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:36.123072 31521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:14:36.123097 31521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:36.135479 31521 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:33825
I20260808 02:14:36.136832 31521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:14:36.139566 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 31521
I20260808 02:14:36.139726 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:14:36.140072 19170 external_mini_cluster.cc:1428] Setting key 9f44c4d45cec57c115976cd280f80f4a
I20260808 02:14:36.150238 31664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:33825 every 8 connection(s)
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
Aug 08 02:14:36 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:14:36.191598 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.152549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:55739 (local address 127.18.184.190:46109)
0808 02:14:36.159963 (+ 7414us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:36.159969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:36.159985 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:14:36.160031 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:36.160033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:36.160083 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:36.160241 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:14:36.165023 (+ 4782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.165872 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.175694 (+ 9822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.176067 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.176204 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.183377 (+ 7173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:14:36.183382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:14:36.183385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:14:36.183415 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:14:36.185423 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:36.186062 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:36.186065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:36.186067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:36.186168 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:36.189198 (+ 3030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:36.189202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:36.189204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:36.189436 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:14:36.189523 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:36.191296 (+ 1773us) server_negotiation.cc:299] Negotiation successful
0808 02:14:36.191430 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7317,"thread_start_us":142,"threads_started":1}
I20260808 02:14:36.196981 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.153104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:46109 (local address 127.18.184.130:55739)
0808 02:14:36.158051 (+ 4947us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:36.158142 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:36.159490 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:36.160577 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:36.160587 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:36.164028 (+ 3441us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:14:36.164773 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.164796 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.166035 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.166039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.175466 (+ 9427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.175488 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.176634 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.176639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.177268 (+ 629us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.178135 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:14:36.178158 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:14:36.183116 (+ 4958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:14:36.185573 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:14:36.185579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:14:36.185597 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:14:36.185945 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:14:36.188828 (+ 2883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:14:36.188831 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:14:36.188833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:14:36.189037 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:14:36.189579 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:14:36.189587 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:14:36.189924 (+ 337us) client_negotiation.cc:769] Sending connection context
0808 02:14:36.190119 (+ 195us) client_negotiation.cc:240] Negotiation successful
0808 02:14:36.190530 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4593,"thread_start_us":148,"threads_started":1}
I20260808 02:14:36.199400 31665 heartbeater.cc:347] Connected to a master server at 127.18.184.190:46109
I20260808 02:14:36.199690 31665 heartbeater.cc:464] Registering TS with master...
I20260808 02:14:36.200349 31665 heartbeater.cc:511] Master 127.18.184.190:46109 requested a full tablet report, sending...
I20260808 02:14:36.201678 31286 ts_manager.cc:194] Registered new tserver with Master: e8d59c5411784deeaf1540061d60f8e8 (127.18.184.130:33825)
I20260808 02:14:36.202538 31286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:55739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:14:36.221930 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.184.148:50335/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46109
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:14:36 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
W20260808 02:14:36.420321 31671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:14:36.420694 31671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:14:36.420789 31671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:14:36.425521 31671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:14:36.425597 31671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:14:36.425693 31671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:14:36.433574 31671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.184.148:50335/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:14:36.435134 31671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:14:36.436651 31671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:14:36.451088 31677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:36.451673 31676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:36.451795 31679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:14:36.452744 31671 server_base.cc:1034] running on GCE node
I20260808 02:14:36.453423 31671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:14:36.454246 31671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:14:36.461977 31671 hybrid_clock.cc:648] HybridClock initialized: now 1786155276461892 us; error 100 us; skew 500 ppm
Aug 08 02:14:36 dist-test-slave-9gcw krb5kdc[30773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155276, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:14:36.466281 31671 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:14:36.467494 31671 webserver.cc:460] Webserver started at http://127.18.184.131:46609/ using document root <none> and password file <none>
I20260808 02:14:36.468290 31671 fs_manager.cc:359] Metadata directory not provided
I20260808 02:14:36.468384 31671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:14:36.468636 31671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:14:36 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155276, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:14:36.482999 31513 heartbeater.cc:503] Master 127.18.184.190:46109 was elected leader, sending a full tablet report...
I20260808 02:14:36.518604 31671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "1ade18d2e2b946398d2971ca56bff021"
format_stamp: "Formatted at 2026-08-08 02:14:36 on dist-test-slave-9gcw"
server_key: "51f1508be93a8eda53bfbb4544e56775"
server_key_iv: "59bd3695e8a166e24dca844370ca9331"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:36.519527 31671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "1ade18d2e2b946398d2971ca56bff021"
format_stamp: "Formatted at 2026-08-08 02:14:36 on dist-test-slave-9gcw"
server_key: "51f1508be93a8eda53bfbb4544e56775"
server_key_iv: "59bd3695e8a166e24dca844370ca9331"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:36.544896 31671 fs_manager.cc:691] Time spent creating directory manager: real 0.025s user 0.004s sys 0.000s
I20260808 02:14:36.571887 31687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:36.573825 31671 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.000s
I20260808 02:14:36.573987 31671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "1ade18d2e2b946398d2971ca56bff021"
format_stamp: "Formatted at 2026-08-08 02:14:36 on dist-test-slave-9gcw"
server_key: "51f1508be93a8eda53bfbb4544e56775"
server_key_iv: "59bd3695e8a166e24dca844370ca9331"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:36.574141 31671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:14:36.609632 31671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:14:36.613454 31671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:14:36.613651 31671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:14:36.614372 31671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:14:36.615420 31671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:14:36.615481 31671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:36.615535 31671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:14:36.615561 31671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:36.630322 31671 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:39823
I20260808 02:14:36.630604 31811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:39823 every 8 connection(s)
I20260808 02:14:36.631680 31671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:14:36.638573 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 31671
I20260808 02:14:36.638708 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:14:36.639045 19170 external_mini_cluster.cc:1428] Setting key 7bdb7aa1c310a4f07995916f6ecf4d5f
Aug 08 02:14:36 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155276, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:14:36.654405 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.637024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:46759 (local address 127.18.184.190:46109)
0808 02:14:36.637233 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:36.637238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:36.637765 (+ 527us) server_negotiation.cc:407] Connection header received
0808 02:14:36.639085 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:36.639089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:36.639143 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:36.639203 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:14:36.642163 (+ 2960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.642962 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.645235 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.645613 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.645723 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.648703 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:14:36.648707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:14:36.648710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:14:36.648743 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:14:36.650602 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:36.651247 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:36.651250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:36.651253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:36.651375 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:36.652207 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:36.652210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:36.652212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:36.653121 (+ 909us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:14:36.653266 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:36.654009 (+ 743us) server_negotiation.cc:299] Negotiation successful
0808 02:14:36.654180 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:14:36.655551 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.635676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:46109 (local address 127.18.184.131:46759)
0808 02:14:36.637583 (+ 1907us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:36.637644 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:36.638832 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:36.640933 (+ 2101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:36.640943 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:36.641369 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:14:36.642005 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.642020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.643115 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.643119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.644745 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.644758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.645783 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.645787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.646033 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.646756 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:14:36.646776 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:14:36.648509 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:14:36.650736 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:14:36.650743 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:14:36.650761 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:14:36.651127 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:14:36.651635 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:14:36.651638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:14:36.651641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:14:36.651843 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:14:36.653320 (+ 1477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:14:36.653327 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:14:36.653693 (+ 366us) client_negotiation.cc:769] Sending connection context
0808 02:14:36.653952 (+ 259us) client_negotiation.cc:240] Negotiation successful
0808 02:14:36.654400 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1648,"thread_start_us":215,"threads_started":1}
I20260808 02:14:36.657013 31812 heartbeater.cc:347] Connected to a master server at 127.18.184.190:46109
I20260808 02:14:36.657326 31812 heartbeater.cc:464] Registering TS with master...
I20260808 02:14:36.657958 31812 heartbeater.cc:511] Master 127.18.184.190:46109 requested a full tablet report, sending...
I20260808 02:14:36.659248 31286 ts_manager.cc:194] Registered new tserver with Master: 1ade18d2e2b946398d2971ca56bff021 (127.18.184.131:39823)
I20260808 02:14:36.660043 31286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:46759
I20260808 02:14:36.664899 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:14:36.680137 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.667547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60328 (local address 127.18.184.190:46109)
0808 02:14:36.667669 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:36.667674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:36.667905 (+ 231us) server_negotiation.cc:407] Connection header received
0808 02:14:36.668092 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:36.668096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:36.668144 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:36.668204 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:14:36.669195 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.669976 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.671127 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.671459 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.671529 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.673489 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:14:36.673493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:14:36.673496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:14:36.673526 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:14:36.677840 (+ 4314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:36.678658 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:36.678662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:36.678664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:36.678768 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:36.679142 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:36.679146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:36.679148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:36.679400 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:14:36.679540 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:36.679827 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 02:14:36.679960 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:14:36.684896 31286 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:14:36.689702 31286 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:14:36.716962 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.707765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:33825 (local address 127.0.0.1:41648)
0808 02:14:36.710392 (+ 2627us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:36.710449 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:36.710591 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:36.711638 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:36.711644 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:36.711669 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:14:36.711858 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.711868 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.713330 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.713333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.714714 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.714726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.716332 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.716335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.716452 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.716495 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:14:36.716615 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 02:14:36.716751 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2437,"thread_start_us":126,"threads_started":1}
Aug 08 02:14:36 dist-test-slave-9gcw krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:14:36.717900 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.707893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.18.184.130:33825)
0808 02:14:36.709955 (+ 2062us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:36.709963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:36.710748 (+ 785us) server_negotiation.cc:407] Connection header received
0808 02:14:36.710835 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:36.710839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:36.711082 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:36.711233 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:14:36.712048 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.713186 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.714881 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.715733 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.715880 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.716031 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:36.717635 (+ 1604us) server_negotiation.cc:299] Negotiation successful
0808 02:14:36.717749 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1939,"thread_start_us":90,"threads_started":1}
I20260808 02:14:36.721195 31593 tablet_service.cc:1467] Processing CreateTablet for tablet f14a93a13b594c1394296a780728774b (DEFAULT_TABLE table=test-table [id=6530ad1959de4cc58487a2bb3bfe7a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:14:36.722419 31593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14a93a13b594c1394296a780728774b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:14:36.725561 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.708583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:36497 (local address 127.0.0.1:35940)
0808 02:14:36.717373 (+ 8790us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:36.717386 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:36.717493 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:36.719369 (+ 1876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:36.719371 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:36.719387 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:14:36.719467 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.719473 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.721173 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.721176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.722249 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.722264 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.725120 (+ 2856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.725123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.725237 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.725264 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:14:36.725341 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 02:14:36.725415 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8729,"thread_start_us":101,"threads_started":1}
I20260808 02:14:36.726418 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.709196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35940 (local address 127.18.184.129:36497)
0808 02:14:36.716255 (+ 7059us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:36.716264 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:36.717482 (+ 1218us) server_negotiation.cc:407] Connection header received
0808 02:14:36.717694 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:36.717700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:36.717943 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:36.718267 (+ 324us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:14:36.719669 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.721012 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.723956 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.724778 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.724921 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.725054 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:36.726160 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0808 02:14:36.726268 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6935,"thread_start_us":144,"threads_started":1}
I20260808 02:14:36.735832 31430 tablet_service.cc:1467] Processing CreateTablet for tablet f14a93a13b594c1394296a780728774b (DEFAULT_TABLE table=test-table [id=6530ad1959de4cc58487a2bb3bfe7a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:14:36.736924 31430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14a93a13b594c1394296a780728774b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:14:36.739492 31832 tablet_bootstrap.cc:492] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8: Bootstrap starting.
I20260808 02:14:36.742082 31832 tablet_bootstrap.cc:654] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8: Neither blocks nor log segments found. Creating new log.
I20260808 02:14:36.743311 31832 log.cc:824] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:14:36.745857 31832 tablet_bootstrap.cc:492] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8: No bootstrap required, opened a new log
I20260808 02:14:36.746079 31832 ts_tablet_manager.cc:1397] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:14:36.747340 31834 tablet_bootstrap.cc:492] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd: Bootstrap starting.
I20260808 02:14:36.750157 31834 tablet_bootstrap.cc:654] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd: Neither blocks nor log segments found. Creating new log.
I20260808 02:14:36.751192 31834 log.cc:824] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd: Log is configured to *not* fsync() on all Append() calls
I20260808 02:14:36.751896 31832 raft_consensus.cc:348] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d59c5411784deeaf1540061d60f8e8" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 33825 } } peers { permanent_uuid: "5d8058ffc81845e3aea8b9f29bd833dd" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 36497 } } peers { permanent_uuid: "1ade18d2e2b946398d2971ca56bff021" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 39823 } }
I20260808 02:14:36.752233 31832 raft_consensus.cc:374] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:14:36.752290 31832 raft_consensus.cc:729] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8d59c5411784deeaf1540061d60f8e8, State: Initialized, Role: FOLLOWER
I20260808 02:14:36.752796 31832 consensus_queue.cc:260] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d59c5411784deeaf1540061d60f8e8" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 33825 } } peers { permanent_uuid: "5d8058ffc81845e3aea8b9f29bd833dd" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 36497 } } peers { permanent_uuid: "1ade18d2e2b946398d2971ca56bff021" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 39823 } }
I20260808 02:14:36.754024 31832 ts_tablet_manager.cc:1428] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8: Time spent starting tablet: real 0.008s user 0.006s sys 0.002s
I20260808 02:14:36.754493 31665 heartbeater.cc:503] Master 127.18.184.190:46109 was elected leader, sending a full tablet report...
I20260808 02:14:36.755738 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.711789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.18.184.131:39823)
0808 02:14:36.726832 (+ 15043us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:36.726845 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:36.726872 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:14:36.726947 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:36.726953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:36.727169 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:36.727334 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:14:36.749422 (+ 22088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.750684 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.753093 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.754845 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.754991 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.755134 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:36.755417 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 02:14:36.755551 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14903,"thread_start_us":129,"threads_started":1}
I20260808 02:14:36.756251 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.711361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:39823 (local address 127.0.0.1:35682)
0808 02:14:36.723714 (+ 12353us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:36.723729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:36.723839 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:36.731476 (+ 7637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:36.731481 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:36.731511 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:14:36.731632 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.731639 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.751699 (+ 20060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.751704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.752862 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.752879 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.755014 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.755019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.755174 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.755208 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:14:36.755963 (+ 755us) client_negotiation.cc:240] Negotiation successful
0808 02:14:36.756078 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12282,"thread_start_us":92,"threads_started":1}
I20260808 02:14:36.759894 31738 tablet_service.cc:1467] Processing CreateTablet for tablet f14a93a13b594c1394296a780728774b (DEFAULT_TABLE table=test-table [id=6530ad1959de4cc58487a2bb3bfe7a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:14:36.761196 31738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14a93a13b594c1394296a780728774b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:14:36.765518 31834 tablet_bootstrap.cc:492] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd: No bootstrap required, opened a new log
I20260808 02:14:36.765811 31834 ts_tablet_manager.cc:1397] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.000s
I20260808 02:14:36.771661 31834 raft_consensus.cc:348] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d59c5411784deeaf1540061d60f8e8" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 33825 } } peers { permanent_uuid: "5d8058ffc81845e3aea8b9f29bd833dd" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 36497 } } peers { permanent_uuid: "1ade18d2e2b946398d2971ca56bff021" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 39823 } }
I20260808 02:14:36.771974 31834 raft_consensus.cc:374] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:14:36.772042 31834 raft_consensus.cc:729] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d8058ffc81845e3aea8b9f29bd833dd, State: Initialized, Role: FOLLOWER
I20260808 02:14:36.772536 31834 consensus_queue.cc:260] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d59c5411784deeaf1540061d60f8e8" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 33825 } } peers { permanent_uuid: "5d8058ffc81845e3aea8b9f29bd833dd" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 36497 } } peers { permanent_uuid: "1ade18d2e2b946398d2971ca56bff021" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 39823 } }
I20260808 02:14:36.773888 31834 ts_tablet_manager.cc:1428] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260808 02:14:36.774315 31837 tablet_bootstrap.cc:492] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021: Bootstrap starting.
I20260808 02:14:36.782245 31837 tablet_bootstrap.cc:654] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021: Neither blocks nor log segments found. Creating new log.
I20260808 02:14:36.784514 31837 log.cc:824] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021: Log is configured to *not* fsync() on all Append() calls
I20260808 02:14:36.790577 31837 tablet_bootstrap.cc:492] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021: No bootstrap required, opened a new log
I20260808 02:14:36.790840 31837 ts_tablet_manager.cc:1397] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.000s
I20260808 02:14:36.797413 31837 raft_consensus.cc:348] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d59c5411784deeaf1540061d60f8e8" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 33825 } } peers { permanent_uuid: "5d8058ffc81845e3aea8b9f29bd833dd" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 36497 } } peers { permanent_uuid: "1ade18d2e2b946398d2971ca56bff021" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 39823 } }
I20260808 02:14:36.797780 31837 raft_consensus.cc:374] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:14:36.797856 31837 raft_consensus.cc:729] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ade18d2e2b946398d2971ca56bff021, State: Initialized, Role: FOLLOWER
I20260808 02:14:36.799227 31837 consensus_queue.cc:260] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d59c5411784deeaf1540061d60f8e8" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 33825 } } peers { permanent_uuid: "5d8058ffc81845e3aea8b9f29bd833dd" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 36497 } } peers { permanent_uuid: "1ade18d2e2b946398d2971ca56bff021" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 39823 } }
I20260808 02:14:36.801595 31837 ts_tablet_manager.cc:1428] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260808 02:14:36.802047 31812 heartbeater.cc:503] Master 127.18.184.190:46109 was elected leader, sending a full tablet report...
I20260808 02:14:36.805498 31841 raft_consensus.cc:482] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:14:36.805691 31841 raft_consensus.cc:504] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d59c5411784deeaf1540061d60f8e8" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 33825 } } peers { permanent_uuid: "5d8058ffc81845e3aea8b9f29bd833dd" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 36497 } } peers { permanent_uuid: "1ade18d2e2b946398d2971ca56bff021" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 39823 } }
I20260808 02:14:36.810575 31841 leader_election.cc:288] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8d59c5411784deeaf1540061d60f8e8 (127.18.184.130:33825), 5d8058ffc81845e3aea8b9f29bd833dd (127.18.184.129:36497)
I20260808 02:14:36.812651 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.807889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:33825 (local address 127.18.184.131:33981)
0808 02:14:36.808028 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:36.808045 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:36.808130 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:36.808639 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:36.808642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:36.808662 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:14:36.808787 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.808794 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.809891 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.809895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.811119 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.811130 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.812230 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.812233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.812352 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.812377 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:14:36.812442 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:14:36.812521 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 02:14:36.813372 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.808171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:33981 (local address 127.18.184.130:33825)
0808 02:14:36.808318 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:36.808324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:36.808339 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:14:36.808397 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:36.808400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:36.808445 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:36.808517 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:14:36.808944 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.809739 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.811330 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.811979 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.812104 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.812161 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:36.813132 (+ 971us) server_negotiation.cc:299] Negotiation successful
0808 02:14:36.813255 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:14:36.813970 31609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14a93a13b594c1394296a780728774b" candidate_uuid: "1ade18d2e2b946398d2971ca56bff021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8d59c5411784deeaf1540061d60f8e8" is_pre_election: true
I20260808 02:14:36.814311 31609 raft_consensus.cc:2454] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ade18d2e2b946398d2971ca56bff021 in term 0.
I20260808 02:14:36.815135 31691 leader_election.cc:302] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ade18d2e2b946398d2971ca56bff021, e8d59c5411784deeaf1540061d60f8e8; no voters:
I20260808 02:14:36.815629 31841 raft_consensus.cc:2781] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:14:36.815742 31841 raft_consensus.cc:482] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:14:36.815812 31841 raft_consensus.cc:3037] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:14:36.817546 31841 raft_consensus.cc:504] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d59c5411784deeaf1540061d60f8e8" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 33825 } } peers { permanent_uuid: "5d8058ffc81845e3aea8b9f29bd833dd" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 36497 } } peers { permanent_uuid: "1ade18d2e2b946398d2971ca56bff021" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 39823 } }
I20260808 02:14:36.818388 31609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14a93a13b594c1394296a780728774b" candidate_uuid: "1ade18d2e2b946398d2971ca56bff021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8d59c5411784deeaf1540061d60f8e8"
I20260808 02:14:36.818540 31609 raft_consensus.cc:3037] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:14:36.820119 31609 raft_consensus.cc:2454] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ade18d2e2b946398d2971ca56bff021 in term 1.
I20260808 02:14:36.820642 31691 leader_election.cc:302] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ade18d2e2b946398d2971ca56bff021, e8d59c5411784deeaf1540061d60f8e8; no voters:
I20260808 02:14:36.821136 31841 leader_election.cc:288] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [CANDIDATE]: Term 1 election: Requested vote from peers e8d59c5411784deeaf1540061d60f8e8 (127.18.184.130:33825), 5d8058ffc81845e3aea8b9f29bd833dd (127.18.184.129:36497)
I20260808 02:14:36.821282 31841 raft_consensus.cc:2781] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:14:36.821377 31841 raft_consensus.cc:686] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 1 LEADER]: Becoming Leader. State: Replica: 1ade18d2e2b946398d2971ca56bff021, State: Running, Role: LEADER
I20260808 02:14:36.821909 31841 consensus_queue.cc:237] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d59c5411784deeaf1540061d60f8e8" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 33825 } } peers { permanent_uuid: "5d8058ffc81845e3aea8b9f29bd833dd" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 36497 } } peers { permanent_uuid: "1ade18d2e2b946398d2971ca56bff021" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 39823 } }
I20260808 02:14:36.822706 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.810427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:39117 (local address 127.18.184.129:36497)
0808 02:14:36.810541 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:36.810545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:36.811512 (+ 967us) server_negotiation.cc:407] Connection header received
0808 02:14:36.811704 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:36.811708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:36.811762 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:36.811813 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:14:36.814115 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.814876 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.819360 (+ 4484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.819969 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.820105 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.820159 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:36.822467 (+ 2308us) server_negotiation.cc:299] Negotiation successful
0808 02:14:36.822574 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:14:36.823596 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.809989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:36497 (local address 127.18.184.131:39117)
0808 02:14:36.811423 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:36.811440 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:36.811583 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:36.811954 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:36.811956 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:36.811976 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:14:36.813955 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.813964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.817268 (+ 3304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.817272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.819192 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:36.819204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.821022 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.821026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:36.821137 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.821210 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 02:14:36.823323 (+ 2113us) client_negotiation.cc:240] Negotiation successful
0808 02:14:36.823416 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"thread_start_us":133,"threads_started":1}
I20260808 02:14:36.824520 31448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14a93a13b594c1394296a780728774b" candidate_uuid: "1ade18d2e2b946398d2971ca56bff021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d8058ffc81845e3aea8b9f29bd833dd" is_pre_election: true
I20260808 02:14:36.825073 31448 raft_consensus.cc:2454] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ade18d2e2b946398d2971ca56bff021 in term 0.
I20260808 02:14:36.825698 31449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14a93a13b594c1394296a780728774b" candidate_uuid: "1ade18d2e2b946398d2971ca56bff021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d8058ffc81845e3aea8b9f29bd833dd"
I20260808 02:14:36.825822 31449 raft_consensus.cc:3037] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:14:36.827231 31449 raft_consensus.cc:2454] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ade18d2e2b946398d2971ca56bff021 in term 1.
I20260808 02:14:36.829566 31285 catalog_manager.cc:5495] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ade18d2e2b946398d2971ca56bff021 (127.18.184.131). New cstate: current_term: 1 leader_uuid: "1ade18d2e2b946398d2971ca56bff021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d59c5411784deeaf1540061d60f8e8" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 33825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d8058ffc81845e3aea8b9f29bd833dd" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 36497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ade18d2e2b946398d2971ca56bff021" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 39823 } health_report { overall_health: HEALTHY } } }
I20260808 02:14:36.848683 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.843719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35698 (local address 127.18.184.131:39823)
0808 02:14:36.843870 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:36.843875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:36.843890 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:14:36.843986 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:36.843989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:36.844057 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:36.844168 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:14:36.844676 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.845572 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.846816 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:36.847127 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:36.847254 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:36.847630 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:14:36.848236 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:14:36.848361 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:36.848460 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 02:14:36.848530 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:14:36.856680 31449 raft_consensus.cc:1261] T f14a93a13b594c1394296a780728774b P 5d8058ffc81845e3aea8b9f29bd833dd [term 1 FOLLOWER]: Refusing update from remote peer 1ade18d2e2b946398d2971ca56bff021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:14:36.858584 31609 raft_consensus.cc:1261] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8 [term 1 FOLLOWER]: Refusing update from remote peer 1ade18d2e2b946398d2971ca56bff021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:14:36.859259 31841 consensus_queue.cc:1035] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d8058ffc81845e3aea8b9f29bd833dd" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 36497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:14:36.865868 31841 consensus_queue.cc:1035] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8d59c5411784deeaf1540061d60f8e8" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:14:36.876449 31847 mvcc.cc:205] Tried to move back new op lower bound from 7316092013994233856 to 7316092013866606592. Current Snapshot: MvccSnapshot[applied={T|T < 7316092013994233856}]
I20260808 02:14:36.881529 31850 mvcc.cc:205] Tried to move back new op lower bound from 7316092013994233856 to 7316092013866606592. Current Snapshot: MvccSnapshot[applied={T|T < 7316092013994233856}]
I20260808 02:14:36.888797 31852 mvcc.cc:205] Tried to move back new op lower bound from 7316092013994233856 to 7316092013866606592. Current Snapshot: MvccSnapshot[applied={T|T < 7316092013994233856}]
I20260808 02:14:36.907254 31285 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:14:36.907485 31285 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:14:36.910701 31285 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 051c89e71f204680aefc6608d7f750b5: Sending DeleteTablet for 3 replicas of tablet f14a93a13b594c1394296a780728774b
I20260808 02:14:36.912635 31593 tablet_service.cc:1514] Processing DeleteTablet for tablet f14a93a13b594c1394296a780728774b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:14:36 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:41648
I20260808 02:14:36.913321 31430 tablet_service.cc:1514] Processing DeleteTablet for tablet f14a93a13b594c1394296a780728774b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:14:36 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:35940
I20260808 02:14:36.913877 31738 tablet_service.cc:1514] Processing DeleteTablet for tablet f14a93a13b594c1394296a780728774b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:14:36 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:35682
I20260808 02:14:36.915038 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 31372
I20260808 02:14:36.918572 31859 tablet_replica.cc:330] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8: stopping tablet replica
I20260808 02:14:36.918978 31859 raft_consensus.cc:2229] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:14:36.919269 31859 raft_consensus.cc:2258] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:14:36.919641 31861 tablet_replica.cc:330] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021: stopping tablet replica
I20260808 02:14:36.919937 31861 raft_consensus.cc:2229] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:14:36.920282 31861 raft_consensus.cc:2258] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:14:36.938366 31861 ts_tablet_manager.cc:1905] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:14:36.940688 31859 ts_tablet_manager.cc:1905] T f14a93a13b594c1394296a780728774b P e8d59c5411784deeaf1540061d60f8e8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 02:14:36.941113 31270 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:36497 (error 108)
I20260808 02:14:36.943182 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 31521
I20260808 02:14:36.952363 31861 ts_tablet_manager.cc:1918] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:14:36.952524 31861 log.cc:1196] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/f14a93a13b594c1394296a780728774b
I20260808 02:14:36.952977 31861 ts_tablet_manager.cc:1939] T f14a93a13b594c1394296a780728774b P 1ade18d2e2b946398d2971ca56bff021: Deleting consensus metadata
I20260808 02:14:36.953814 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.953393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:36497 (local address 127.0.0.1:35944)
0808 02:14:36.953609 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:36.953700 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:36497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260808 02:14:36.954150 31270 catalog_manager.cc:4584] TS 5d8058ffc81845e3aea8b9f29bd833dd (127.18.184.129:36497): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f14a93a13b594c1394296a780728774b: Network error: Client connection negotiation failed: client connection to 127.18.184.129:36497: connect: Connection refused (error 111)
I20260808 02:14:36.954680 31270 catalog_manager.cc:4841] TS 1ade18d2e2b946398d2971ca56bff021 (127.18.184.131:39823): tablet f14a93a13b594c1394296a780728774b (table test-table [id=6530ad1959de4cc58487a2bb3bfe7a7d]) successfully deleted
I20260808 02:14:36.965929 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 31671
I20260808 02:14:36.973690 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:36.973270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:33825 (local address 127.0.0.1:41656)
0808 02:14:36.973510 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:36.973580 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.130:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260808 02:14:36.982985 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 31251
I20260808 02:14:36.999492 19170 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 02:14:37.345103 19170 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 02:14:37.345384 19170 mini_ranger.cc:66] Stopping Ranger...
I20260808 02:14:37.701074 19170 mini_ranger.cc:68] Stopped Ranger
2026-08-08 02:14:37.701 UTC [30804] LOG: received smart shutdown request
2026-08-08 02:14:37.705 UTC [30804] LOG: background worker "logical replication launcher" (PID 30814) exited with exit code 1
2026-08-08 02:14:37.706 UTC [30809] LOG: shutting down
2026-08-08 02:14:37.906 UTC [30804] LOG: database system is shut down
2026-08-08T02:14:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50690 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:14:38 dist-test-slave-9gcw krb5kdc[31872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:14:38 dist-test-slave-9gcw krb5kdc[31872](info): set up 2 sockets
Aug 08 02:14:38 dist-test-slave-9gcw krb5kdc[31872](info): commencing operation
krb5kdc: starting...
W20260808 02:14:39.142884 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:14:39 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:14:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:14:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.184.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.184.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/rangeradmin_127.18.184.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/rangeradmin_127.18.184.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.184.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.184.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/rangerlookup_127.18.184.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/rangerlookup_127.18.184.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.184.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.184.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/HTTP_127.18.184.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/HTTP_127.18.184.148@KRBTEST.COM.keytab.
I20260808 02:14:39.344584 19170 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmhslVu/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/postgres -l logfile start
2026-08-08 02:14:42.129 UTC [31903] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 02:14:42.129 UTC [31903] LOG: listening on IPv4 address "127.18.184.148", port 41171
2026-08-08 02:14:42.134 UTC [31903] LOG: listening on Unix socket "/tmp/.s.PGSQL.41171"
2026-08-08 02:14:42.158 UTC [31906] LOG: database system was shut down at 2026-08-08 02:14:40 UTC
2026-08-08 02:14:42.162 UTC [31903] LOG: database system is ready to accept connections
I20260808 02:14:43.196610 19170 mini_postgres.cc:96] Postgres bound to 41171
2026-08-08 02:14:43.204 UTC [31914] FATAL: database "slave" does not exist
127.18.184.148:41171 - accepting connections
I20260808 02:14:43.205077 19170 mini_ranger.cc:161] Starting Ranger...
I20260808 02:14:43.222684 19170 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 02:14:43.779462 19170 mini_ranger.cc:87] Created ranger Postgres database
I20260808 02:14:43.779569 19170 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-admin
2026-08-08 02:14:44,078 [I] DB FLAVOR :POSTGRES
2026-08-08 02:14:44,079 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:14:44,079 [I] Checking connection..
2026-08-08 02:14:44,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 02:14:44,355 [I] Checking connection passed.
2026-08-08 02:14:44,355 [I] --------- Verifying version history table ---------
2026-08-08 02:14:44,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:14:44,636 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:14:44,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:14:44,913 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:14:44,913 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 02:14:44,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 02:14:45.193 UTC [31985] WARNING: there is no transaction in progress
2026-08-08 02:14:45,201 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 02:14:45,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:14:45,497 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 02:14:45,497 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:14:45,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 02:14:45,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 02:14:46,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9gcw.c.gcp-upstream.internal','N') ;"
2026-08-08 02:14:46,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:14:46,570 [I] Table x_portal_user does not exist in database ranger
2026-08-08 02:14:46,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:14:46,841 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 02:14:46,841 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:14:47,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:14:47,391 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 02:14:47,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 02:14:48.502 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.520 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.539 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.557 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.577 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.598 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.671 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.679 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.697 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.709 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.721 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.736 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.748 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.756 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.765 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:48.774 UTC [32103] WARNING: there is no transaction in progress
2026-08-08 02:14:49,513 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 02:14:49,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 02:14:49,757 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 02:14:49,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:14:50,022 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 02:14:50,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:14:50,270 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 02:14:50,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:14:50,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:14:50,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9gcw.c.gcp-upstream.internal';"
2026-08-08 02:14:51,090 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 02:14:51,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 02:14:51,343 [I] DB_PATCHES have already been applied
I20260808 02:14:51.349975 19170 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmhslVu/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 02:14:51.350056 19170 mini_ranger.cc:193] Using host: 127.18.184.148
I20260808 02:14:51.355075 19170 mini_ranger.cc:227] Ranger admin URL: http://127.18.184.148:44017
Aug 08, 2026 2:14:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:14:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:14:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:14:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:14:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:14:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:14:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 02:14:52.559932 19170 mini_ranger.cc:160] Time spent starting Ranger: real 9.355s user 0.004s sys 0.009s
Aug 08, 2026 2:15:13 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:15:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 2:15:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:15:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:15:13 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:15:14 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 02:15:14.800530 19170 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.184.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.184.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/rangerkms_127.18.184.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.184.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/rangerkms_127.18.184.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.184.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.184.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.184.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/HTTP_127.18.184.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.184.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/HTTP_127.18.184.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 02:15:14 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155314, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 02:15:14.900530 19170 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 02:15:14.913604 19170 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 02:15:15.894905 19170 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 02:15:15.895016 19170 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-kms
I20260808 02:15:15.895037 19170 mini_ranger_kms.cc:227] Using postgres at 127.18.184.148:41171
2026-08-08 02:15:16,096 [I] DB FLAVOR :POSTGRES
2026-08-08 02:15:16,096 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:15:16,097 [I] Checking connection
2026-08-08 02:15:16,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 02:15:16,384 [I] connection success
2026-08-08 02:15:16,384 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 02:15:16,384 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 02:15:16,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 02:15:16,648 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 02:15:16,649 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:15:16,649 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 02:15:16,649 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.184.148:41171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 02:15:16,948 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 02:15:17.500847 19170 mini_ranger_kms.cc:310] Created kms service
I20260808 02:15:17.724184 19170 mini_ranger_kms.cc:326] Created kudu user
I20260808 02:15:17.772884 19170 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 02:15:17 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155314, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:15:18.259539 19170 mini_ranger_kms.cc:384] Added ranger policy
I20260808 02:15:18.259812 19170 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmhslVu/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmhslVu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 02:15:18.259840 19170 mini_ranger_kms.cc:240] Using host: 127.18.184.148
I20260808 02:15:18.265980 19170 mini_ranger_kms.cc:276] Ranger KMS PID: 32296
I20260808 02:15:18.266085 19170 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.184.148:41519
Aug 08, 2026 2:15:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:15:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:15:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:15:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:15:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:15:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 02:15:20 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.184.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:20.704044 19170 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.804s user 0.008s sys 0.178s
I20260808 02:15:20.704177 19170 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 02:15:20.704252 19170 mini_ranger_kms.cc:401] 127.18.184.148:41519/kms/v1/keys
Aug 08 02:15:21 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.184.148@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
Aug 08 02:15:21 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.184.148@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
Aug 08 02:15:24 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:15:24.117094 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34717
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:34717
--ranger_config_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.184.148:41519/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:24.259835 32348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:24.260218 32348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:24.260342 32348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:24.265448 32348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:24.265548 32348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:24.265590 32348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:24.265658 32348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:24.265705 32348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:24.272382 32348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.184.148:41519/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:34717
--ranger_config_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:24.273798 32348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:24.274999 32348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:24.284888 32353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:24.284919 32357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:24.284914 32354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:24.285300 32348 server_base.cc:1034] running on GCE node
I20260808 02:15:24.286027 32348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:24.287180 32348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:24.288383 32348 hybrid_clock.cc:648] HybridClock initialized: now 1786155324288357 us; error 43 us; skew 500 ppm
Aug 08 02:15:24 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155324, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:24.296218 32348 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:15:24.297448 32348 webserver.cc:460] Webserver started at http://127.18.184.190:42747/ using document root <none> and password file <none>
I20260808 02:15:24.298066 32348 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:24.298151 32348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:24.298409 32348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:15:24 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155324, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:15:24.359056 32348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "97df1e753be44a038cda01d1a0b9642a"
format_stamp: "Formatted at 2026-08-08 02:15:24 on dist-test-slave-9gcw"
server_key: "2a5372b5732b4b334989a4b798cfe5c3"
server_key_iv: "6550b23968c3355db20b699f68935b61"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:24.359920 32348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "97df1e753be44a038cda01d1a0b9642a"
format_stamp: "Formatted at 2026-08-08 02:15:24 on dist-test-slave-9gcw"
server_key: "2a5372b5732b4b334989a4b798cfe5c3"
server_key_iv: "6550b23968c3355db20b699f68935b61"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:24.364219 32348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 02:15:24.381062 32364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:24.382718 32348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 02:15:24.382884 32348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "97df1e753be44a038cda01d1a0b9642a"
format_stamp: "Formatted at 2026-08-08 02:15:24 on dist-test-slave-9gcw"
server_key: "2a5372b5732b4b334989a4b798cfe5c3"
server_key_iv: "6550b23968c3355db20b699f68935b61"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:24.383044 32348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:24.408125 32348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:24.411667 32348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:24.411911 32348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:24.421129 32348 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:34717
I20260808 02:15:24.421180 32422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:34717 every 8 connection(s)
I20260808 02:15:24.422292 32348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:24.425534 32423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:24.428165 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 32348
I20260808 02:15:24.428314 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:15:24.428648 19170 external_mini_cluster.cc:1428] Setting key 0079589f5901611963a38e9db2e5cfe9
I20260808 02:15:24.433602 32423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a: Bootstrap starting.
I20260808 02:15:24.436789 32423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:24.437870 32423 log.cc:824] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:24.440461 32423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a: No bootstrap required, opened a new log
Aug 08 02:15:24 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:24.445104 32423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97df1e753be44a038cda01d1a0b9642a" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34717 } }
I20260808 02:15:24.445461 32423 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:24.445560 32423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97df1e753be44a038cda01d1a0b9642a, State: Initialized, Role: FOLLOWER
I20260808 02:15:24.446123 32423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97df1e753be44a038cda01d1a0b9642a" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34717 } }
I20260808 02:15:24.446311 32423 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:24.446405 32423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:24.446513 32423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:24.447325 32426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:24.431066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57652 (local address 127.18.184.190:34717)
0808 02:15:24.431606 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:24.431627 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:24.431676 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:15:24.432619 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:24.432645 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:24.433119 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:24.433566 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:24.434520 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:24.436175 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:24.437547 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:24.438347 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:24.438528 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:24.441614 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:24.441620 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:24.441635 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:24.441673 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:24.443552 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:24.444101 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:24.444107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:24.444113 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:24.444236 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:24.444563 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:24.444568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:24.444570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:24.445040 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:24.445310 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:24.446081 (+ 771us) server_negotiation.cc:299] Negotiation successful
0808 02:15:24.446410 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":154,"threads_started":1}
I20260808 02:15:24.447790 32423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97df1e753be44a038cda01d1a0b9642a" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34717 } }
I20260808 02:15:24.448201 32423 leader_election.cc:302] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97df1e753be44a038cda01d1a0b9642a; no voters:
I20260808 02:15:24.448590 32423 leader_election.cc:288] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:15:24.449129 32430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:24.449440 32430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 1 LEADER]: Becoming Leader. State: Replica: 97df1e753be44a038cda01d1a0b9642a, State: Running, Role: LEADER
I20260808 02:15:24.449836 32430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97df1e753be44a038cda01d1a0b9642a" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34717 } }
I20260808 02:15:24.450183 32423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:24.451354 32431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97df1e753be44a038cda01d1a0b9642a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97df1e753be44a038cda01d1a0b9642a" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34717 } } }
I20260808 02:15:24.451354 32430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97df1e753be44a038cda01d1a0b9642a. Latest consensus state: current_term: 1 leader_uuid: "97df1e753be44a038cda01d1a0b9642a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97df1e753be44a038cda01d1a0b9642a" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34717 } } }
I20260808 02:15:24.451522 32431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:24.451545 32430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:24.451969 32438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:24.455376 32438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:24.455835 32423 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 02:15:24.471691 32438 catalog_manager.cc:1347] Generated new cluster ID: ac90c929fbc34ab68413c4ac6ee6f5ca
I20260808 02:15:24.471923 32438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:15:24.507287 32438 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:15:24.508602 32438 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:24.515244 32438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a: Generated new TSK 0
I20260808 02:15:24.516194 32438 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:15:25.272909 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.184.148:41519/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34717
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:25.426398 32470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:25.426872 32470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:25.426987 32470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:25.434121 32470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:25.434307 32470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:25.434463 32470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:15:25.443691 32470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.184.148:41519/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:25.445655 32470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:25.447343 32470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:25.460639 32481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:25.461051 32470 server_base.cc:1034] running on GCE node
W20260808 02:15:25.460705 32479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:25.460997 32478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:25.461892 32470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:25.462584 32470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:25.463792 32470 hybrid_clock.cc:648] HybridClock initialized: now 1786155325463761 us; error 54 us; skew 500 ppm
Aug 08 02:15:25 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:25.470356 32470 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:15:25.471501 32470 webserver.cc:460] Webserver started at http://127.18.184.129:37435/ using document root <none> and password file <none>
I20260808 02:15:25.472170 32470 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:25.472275 32470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:25.472517 32470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:15:25 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:15:25.489326 32470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "9606e43bc30248cd95f57b710b011f44"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-9gcw"
server_key: "992b4daede09a542446d411cdec8d791"
server_key_iv: "b9a22aa58d6f359f675d994b284d39c6"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:25.490007 32470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "9606e43bc30248cd95f57b710b011f44"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-9gcw"
server_key: "992b4daede09a542446d411cdec8d791"
server_key_iv: "b9a22aa58d6f359f675d994b284d39c6"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:25.494288 32470 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:15:25.529767 32489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.531746 32470 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.002s sys 0.001s
I20260808 02:15:25.531997 32470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "9606e43bc30248cd95f57b710b011f44"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-9gcw"
server_key: "992b4daede09a542446d411cdec8d791"
server_key_iv: "b9a22aa58d6f359f675d994b284d39c6"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:25.532258 32470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:25.553028 32470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:25.557130 32470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:25.557451 32470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:25.558230 32470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:25.559350 32470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:25.559461 32470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.559597 32470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:25.559695 32470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.573367 32470 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:45077
I20260808 02:15:25.573436 32613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:45077 every 8 connection(s)
I20260808 02:15:25.574653 32470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:15:25.582551 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 32470
I20260808 02:15:25.582875 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:15:25.583483 19170 external_mini_cluster.cc:1428] Setting key b3016784f4238f686e476b36f4e2fdbb
Aug 08 02:15:25 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:25.597767 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.578956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:43101 (local address 127.18.184.190:34717)
0808 02:15:25.580150 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.580156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.580174 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:15:25.580232 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.580235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.580292 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.582260 (+ 1968us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:25.582425 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.583453 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.586543 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.587194 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.588446 (+ 1252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.590995 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:25.591000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:25.591005 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:25.591042 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:25.594843 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:25.595574 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:25.595578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:25.595580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:25.595701 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:25.596186 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:25.596190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:25.596192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:25.596476 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:25.596567 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.597385 (+ 818us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.597550 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1062,"thread_start_us":74,"threads_started":1}
I20260808 02:15:25.599900 32616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.577582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34717 (local address 127.18.184.129:43101)
0808 02:15:25.578401 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.578467 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:25.579607 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:25.580674 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:25.580684 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:25.581100 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:25.582059 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.582085 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.583631 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.583637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.586280 (+ 2643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.586318 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.587502 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.587509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.587862 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.588792 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:25.588814 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:25.590768 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:25.595002 (+ 4234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:25.595009 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:25.595028 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:25.595429 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:25.595837 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:25.595840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:25.595842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:25.596057 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:25.596643 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:25.596651 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:25.597020 (+ 369us) client_negotiation.cc:769] Sending connection context
0808 02:15:25.598323 (+ 1303us) client_negotiation.cc:240] Negotiation successful
0808 02:15:25.598733 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":164,"threads_started":1}
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:15:25.601977 32614 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34717
I20260808 02:15:25.602181 32614 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:25.602835 32614 heartbeater.cc:511] Master 127.18.184.190:34717 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
I20260808 02:15:25.608917 32383 ts_manager.cc:194] Registered new tserver with Master: 9606e43bc30248cd95f57b710b011f44 (127.18.184.129:45077)
I20260808 02:15:25.610634 32383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:43101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:15:25.638350 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.184.148:41519/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34717
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:15:25 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 02:15:25.820220 32620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:25.820645 32620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:25.820724 32620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:25.827086 32620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:25.827198 32620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:25.827311 32620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:15:25.836066 32620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.184.148:41519/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:25.837689 32620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:25.839273 32620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:25.855768 32632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:25.855788 32626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:25.855945 32628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:25.856423 32620 server_base.cc:1034] running on GCE node
I20260808 02:15:25.857043 32620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:25.858036 32620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:25.859407 32620 hybrid_clock.cc:648] HybridClock initialized: now 1786155325859334 us; error 94 us; skew 500 ppm
Aug 08 02:15:25 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:25.863420 32620 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:15:25.864768 32620 webserver.cc:460] Webserver started at http://127.18.184.130:45675/ using document root <none> and password file <none>
I20260808 02:15:25.865599 32620 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:25.865718 32620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:25.865981 32620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:15:25 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:15:25.885669 32620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "d2fbde563b864f92b85c575197887bf2"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-9gcw"
server_key: "0998391904fa4f863a0d051a9bd244bc"
server_key_iv: "dcbf7e2fb58bc0a74262e0f2e6bab20a"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:25.886705 32620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "d2fbde563b864f92b85c575197887bf2"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-9gcw"
server_key: "0998391904fa4f863a0d051a9bd244bc"
server_key_iv: "dcbf7e2fb58bc0a74262e0f2e6bab20a"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:25.892757 32620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:15:25.913082 32639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.915143 32620 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260808 02:15:25.915302 32620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "d2fbde563b864f92b85c575197887bf2"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-9gcw"
server_key: "0998391904fa4f863a0d051a9bd244bc"
server_key_iv: "dcbf7e2fb58bc0a74262e0f2e6bab20a"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:25.915508 32620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:25.942826 32620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:25.954241 32620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:25.954463 32620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:25.955206 32620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:25.956423 32620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:25.956489 32620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.956545 32620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:25.956578 32620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.973872 32620 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:46219
I20260808 02:15:25.975138 32620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:15:25.979921 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 32620
I20260808 02:15:25.980043 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:15:25.980367 19170 external_mini_cluster.cc:1428] Setting key 23b213332ed065ac10272f30b1f86e96
Aug 08 02:15:25 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:26.002476 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.979917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:60981 (local address 127.18.184.190:34717)
0808 02:15:25.980070 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.980075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.987212 (+ 7137us) server_negotiation.cc:407] Connection header received
0808 02:15:25.988519 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.988523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.988589 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.988674 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:25.990609 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.991553 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.994125 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.994469 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.994554 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.998013 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:25.998017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:25.998020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:25.998052 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:25.999764 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.000441 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.000444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.000446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.000534 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.000925 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.000929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.000930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.001216 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:26.001318 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.002126 (+ 808us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.002294 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:15:26.004374 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.978653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34717 (local address 127.18.184.130:60981)
0808 02:15:25.986967 (+ 8314us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.987065 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:25.988249 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:25.989095 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:25.989108 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:25.989588 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:25.990381 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.990412 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.991740 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.991747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.993956 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.993972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.994815 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.994819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.995020 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.995913 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:25.995937 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:25.997756 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:25.999917 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:25.999924 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:25.999942 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:26.000320 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:26.000636 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:26.000638 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:26.000640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:26.000821 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:26.001394 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:26.001402 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:26.001777 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 02:15:26.002730 (+ 953us) client_negotiation.cc:240] Negotiation successful
0808 02:15:26.003105 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8005,"thread_start_us":176,"threads_started":1}
I20260808 02:15:26.005828 32760 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34717
I20260808 02:15:26.006090 32760 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:26.006729 32760 heartbeater.cc:511] Master 127.18.184.190:34717 requested a full tablet report, sending...
I20260808 02:15:26.007966 32383 ts_manager.cc:194] Registered new tserver with Master: d2fbde563b864f92b85c575197887bf2 (127.18.184.130:46219)
I20260808 02:15:26.008728 32383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:60981
I20260808 02:15:26.009280 32759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:46219 every 8 connection(s)
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:15:26.056773 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.184.148:41519/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34717
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:15:26 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
W20260808 02:15:26.244359 32765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:26.244747 32765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:26.244822 32765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:26.251358 32765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:26.251495 32765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:26.251605 32765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:15:26.260130 32765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.184.148:41519/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:26.261660 32765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:26.263073 32765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:26.277516 304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.278035 306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.278685 303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:26.279395 32765 server_base.cc:1034] running on GCE node
I20260808 02:15:26.280069 32765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:26.280982 32765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:26.282241 32765 hybrid_clock.cc:648] HybridClock initialized: now 1786155326282174 us; error 94 us; skew 500 ppm
Aug 08 02:15:26 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:26.286268 32765 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:15:26.287523 32765 webserver.cc:460] Webserver started at http://127.18.184.131:39263/ using document root <none> and password file <none>
I20260808 02:15:26.288290 32765 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:26.288385 32765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:26.288645 32765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:15:26 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:15:26.316440 32765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "cdc717c386254cbbb3afa9025b523397"
format_stamp: "Formatted at 2026-08-08 02:15:26 on dist-test-slave-9gcw"
server_key: "ba752616ba2dcb39c7c7e28edfb2d59e"
server_key_iv: "48857c7c771e93a5ba9f6cb2c580be61"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:26.317341 32765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "cdc717c386254cbbb3afa9025b523397"
format_stamp: "Formatted at 2026-08-08 02:15:26 on dist-test-slave-9gcw"
server_key: "ba752616ba2dcb39c7c7e28edfb2d59e"
server_key_iv: "48857c7c771e93a5ba9f6cb2c580be61"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:26.324780 32765 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.001s
I20260808 02:15:26.346094 315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.347800 32765 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.003s
I20260808 02:15:26.347939 32765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "cdc717c386254cbbb3afa9025b523397"
format_stamp: "Formatted at 2026-08-08 02:15:26 on dist-test-slave-9gcw"
server_key: "ba752616ba2dcb39c7c7e28edfb2d59e"
server_key_iv: "48857c7c771e93a5ba9f6cb2c580be61"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:26.348083 32765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:26.369899 32765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:26.373646 32765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:26.373842 32765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:26.374539 32765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:26.375636 32765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:26.375698 32765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.375758 32765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:26.375788 32765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.386801 32765 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:32963
I20260808 02:15:26.388347 32765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:15:26.389560 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 32765
I20260808 02:15:26.389685 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:15:26.390048 19170 external_mini_cluster.cc:1428] Setting key 905f0c3c9007e113ededc8a4f598ffb4
I20260808 02:15:26.391373 437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:32963 every 8 connection(s)
Aug 08 02:15:26 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:26.416016 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.391765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:35295 (local address 127.18.184.190:34717)
0808 02:15:26.391901 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:26.391906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:26.400971 (+ 9065us) server_negotiation.cc:407] Connection header received
0808 02:15:26.402739 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:26.402743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:26.402808 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:26.402868 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:26.404639 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.405526 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.407308 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.407668 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.407745 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.411027 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:26.411031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:26.411034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:26.411066 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:26.413240 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.413974 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.413978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.413980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.414085 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.414559 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.414562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.414564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.414822 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:26.414902 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.415698 (+ 796us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.415843 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 02:15:26.417781 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.392358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34717 (local address 127.18.184.131:35295)
0808 02:15:26.400754 (+ 8396us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:26.400840 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:26.402433 (+ 1593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:26.403322 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:26.403331 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:26.403740 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:26.404461 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.404484 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.405706 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.405709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.407164 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.407176 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.407966 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.407970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.408172 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.408945 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:26.408968 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:26.410806 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:26.413400 (+ 2594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:26.413406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:26.413425 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:26.413840 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:26.414219 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:26.414222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:26.414224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:26.414426 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:26.414992 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:26.414998 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:26.415378 (+ 380us) client_negotiation.cc:769] Sending connection context
0808 02:15:26.416268 (+ 890us) client_negotiation.cc:240] Negotiation successful
0808 02:15:26.416656 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8086,"thread_start_us":183,"threads_started":1}
I20260808 02:15:26.419075 438 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34717
I20260808 02:15:26.419337 438 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:26.419943 438 heartbeater.cc:511] Master 127.18.184.190:34717 requested a full tablet report, sending...
I20260808 02:15:26.421054 32383 ts_manager.cc:194] Registered new tserver with Master: cdc717c386254cbbb3afa9025b523397 (127.18.184.131:32963)
I20260808 02:15:26.421865 32383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:35295
I20260808 02:15:26.428048 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:15:26.429045 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 32470
I20260808 02:15:26.440514 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 32620
I20260808 02:15:26.454113 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 32765
I20260808 02:15:26.466300 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 32348
I20260808 02:15:26.481226 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34717
--webserver_interface=127.18.184.190
--webserver_port=42747
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:34717
--ranger_config_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.184.148:41519/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.184.148:41519/kms with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:26.611020 441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:26.611321 441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:26.611362 441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:26.615602 441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:26.615679 441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:26.615698 441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:26.615715 441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:26.615731 441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:26.621718 441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.184.148:41519/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:34717
--ranger_config_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=42747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:26.622902 441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:26.624011 441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:26.633683 448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.633675 450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.633832 447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:26.634222 441 server_base.cc:1034] running on GCE node
I20260808 02:15:26.634838 441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:26.636101 441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:26.637261 441 hybrid_clock.cc:648] HybridClock initialized: now 1786155326637237 us; error 35 us; skew 500 ppm
Aug 08 02:15:26 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:26.640775 441 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:15:26.641872 441 webserver.cc:460] Webserver started at http://127.18.184.190:42747/ using document root <none> and password file <none>
I20260808 02:15:26.642459 441 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:26.642541 441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:15:26 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:15:26.688935 441 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:15:26.691982 459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.693701 441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:26.693836 441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "97df1e753be44a038cda01d1a0b9642a"
format_stamp: "Formatted at 2026-08-08 02:15:24 on dist-test-slave-9gcw"
server_key: "2a5372b5732b4b334989a4b798cfe5c3"
server_key_iv: "6550b23968c3355db20b699f68935b61"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:26.694411 441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:26.715241 441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:26.718510 441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:26.718947 441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:26.727538 441 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:34717
I20260808 02:15:26.727574 515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:34717 every 8 connection(s)
I20260808 02:15:26.728813 441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:26.731431 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 441
I20260808 02:15:26.732059 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.184.148:41519/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:45077
--local_ip_for_outbound_sockets=127.18.184.129
--tserver_master_addrs=127.18.184.190:34717
--webserver_port=37435
--webserver_interface=127.18.184.129
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.184.148:41519/kms with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:26.733764 516 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:15:26.735560 516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a: Bootstrap starting.
I20260808 02:15:26.753396 516 log.cc:824] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:26.760759 516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:15:26.761307 516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a: Bootstrap complete.
I20260808 02:15:26.765849 516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97df1e753be44a038cda01d1a0b9642a" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34717 } }
I20260808 02:15:26.766309 516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97df1e753be44a038cda01d1a0b9642a, State: Initialized, Role: FOLLOWER
I20260808 02:15:26.766840 516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97df1e753be44a038cda01d1a0b9642a" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34717 } }
I20260808 02:15:26.766995 516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:26.767091 516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:26.767216 516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:15:26.769804 516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97df1e753be44a038cda01d1a0b9642a" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34717 } }
I20260808 02:15:26.770152 516 leader_election.cc:302] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97df1e753be44a038cda01d1a0b9642a; no voters:
I20260808 02:15:26.770525 516 leader_election.cc:288] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:15:26.770625 520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:15:26.770840 520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [term 2 LEADER]: Becoming Leader. State: Replica: 97df1e753be44a038cda01d1a0b9642a, State: Running, Role: LEADER
I20260808 02:15:26.771147 520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97df1e753be44a038cda01d1a0b9642a" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34717 } }
I20260808 02:15:26.771867 516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:26.772207 520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97df1e753be44a038cda01d1a0b9642a. Latest consensus state: current_term: 2 leader_uuid: "97df1e753be44a038cda01d1a0b9642a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97df1e753be44a038cda01d1a0b9642a" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34717 } } }
I20260808 02:15:26.772342 520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:26.772745 524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:26.772759 521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "97df1e753be44a038cda01d1a0b9642a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97df1e753be44a038cda01d1a0b9642a" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34717 } } }
I20260808 02:15:26.773046 521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:26.776086 524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:26.776743 516 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 02:15:26.776747 524 catalog_manager.cc:1259] Loaded cluster ID: ac90c929fbc34ab68413c4ac6ee6f5ca
I20260808 02:15:26.778386 524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:15:26.780754 524 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:26.781510 524 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a: Loaded TSK: 0
I20260808 02:15:26.786597 524 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:15:26.884503 518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:26.884899 518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:26.884967 518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:26.891237 518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:26.891332 518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:26.891423 518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:15:26.900048 518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.184.148:41519/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:45077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=37435
--tserver_master_addrs=127.18.184.190:34717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:26.901539 518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:26.902916 518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:26.915732 555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.915733 558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:26.917037 518 server_base.cc:1034] running on GCE node
W20260808 02:15:26.916942 556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:26.917685 518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:26.918541 518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:26.919714 518 hybrid_clock.cc:648] HybridClock initialized: now 1786155326919672 us; error 56 us; skew 500 ppm
Aug 08 02:15:26 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:26.923409 518 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:15:26.924751 518 webserver.cc:460] Webserver started at http://127.18.184.129:37435/ using document root <none> and password file <none>
I20260808 02:15:26.925561 518 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:26.925729 518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:15:26 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:15:26.962174 518 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.000s sys 0.003s
I20260808 02:15:26.967427 567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.970438 518 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.001s
I20260808 02:15:26.970592 518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "9606e43bc30248cd95f57b710b011f44"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-9gcw"
server_key: "992b4daede09a542446d411cdec8d791"
server_key_iv: "b9a22aa58d6f359f675d994b284d39c6"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:26.971407 518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:27.000713 518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:27.004310 518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:27.004539 518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:27.005357 518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:27.006573 518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:27.006640 518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.006705 518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:27.006729 518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.018829 518 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:45077
I20260808 02:15:27.019116 688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:45077 every 8 connection(s)
I20260808 02:15:27.020308 518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:15:27.023551 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 518
I20260808 02:15:27.024048 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.184.148:41519/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:46219
--local_ip_for_outbound_sockets=127.18.184.130
--tserver_master_addrs=127.18.184.190:34717
--webserver_port=45675
--webserver_interface=127.18.184.130
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.184.148:41519/kms with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:15:27 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:27.049957 691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.023084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34717 (local address 127.18.184.129:56169)
0808 02:15:27.026092 (+ 3008us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.026158 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:27.027020 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:27.028265 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:27.028274 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:27.028660 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:27.029414 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:27.029440 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.031431 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.031436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:27.033046 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:27.033061 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.034661 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.034668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:27.034913 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:27.035945 (+ 1032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:27.035970 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:27.037892 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:27.045701 (+ 7809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:27.045711 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:27.045731 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:27.046158 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:27.046717 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:27.046720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:27.046722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:27.046925 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:27.047808 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:27.047815 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:27.048137 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 02:15:27.048352 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 02:15:27.048724 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2759,"thread_start_us":159,"threads_started":1}
I20260808 02:15:27.051841 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.024839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:56169 (local address 127.18.184.190:34717)
0808 02:15:27.025741 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:27.025767 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:27.027277 (+ 1510us) server_negotiation.cc:407] Connection header received
0808 02:15:27.027521 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:27.027530 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:27.027841 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:27.028197 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:27.029648 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.031237 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.033271 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.034134 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.034323 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:27.038209 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:27.038218 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:27.038240 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:27.038284 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:27.045456 (+ 7172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:27.046377 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:27.046385 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:27.046391 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:27.046589 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:27.047063 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:27.047067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:27.047069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:27.047667 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:27.047837 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:27.050562 (+ 2725us) server_negotiation.cc:299] Negotiation successful
0808 02:15:27.050889 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":137,"threads_started":1}
I20260808 02:15:27.054474 689 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34717
I20260808 02:15:27.054757 689 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:27.055413 689 heartbeater.cc:511] Master 127.18.184.190:34717 requested a full tablet report, sending...
W20260808 02:15:27.057999 689 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:34717 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 02:15:27.058151 689 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:27.058375 689 heartbeater.cc:511] Master 127.18.184.190:34717 requested a full tablet report, sending...
I20260808 02:15:27.059477 689 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:27.059686 689 heartbeater.cc:511] Master 127.18.184.190:34717 requested a full tablet report, sending...
W20260808 02:15:27.060604 689 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:15:27.181681 693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:27.182001 693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:27.182047 693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:27.186460 693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:27.186532 693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:27.186595 693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:15:27.193082 693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.184.148:41519/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:46219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=45675
--tserver_master_addrs=127.18.184.190:34717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:27.194653 693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:27.196105 693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:27.209445 701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:27.209832 699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:27.209837 698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:27.210062 693 server_base.cc:1034] running on GCE node
I20260808 02:15:27.210597 693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:27.211357 693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:27.212548 693 hybrid_clock.cc:648] HybridClock initialized: now 1786155327212515 us; error 43 us; skew 500 ppm
Aug 08 02:15:27 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:27.215914 693 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:15:27.217259 693 webserver.cc:460] Webserver started at http://127.18.184.130:45675/ using document root <none> and password file <none>
I20260808 02:15:27.218063 693 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:27.218243 693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:15:27 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:15:27.252882 693 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:15:27.256233 711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.257942 693 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:15:27.258090 693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "d2fbde563b864f92b85c575197887bf2"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-9gcw"
server_key: "0998391904fa4f863a0d051a9bd244bc"
server_key_iv: "dcbf7e2fb58bc0a74262e0f2e6bab20a"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:27.258889 693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:27.287088 693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:27.291503 693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:27.291751 693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:27.292536 693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:27.293740 693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:27.293823 693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.293888 693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:27.293926 693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.305994 693 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:46219
I20260808 02:15:27.306061 832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:46219 every 8 connection(s)
I20260808 02:15:27.307438 693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:15:27.310101 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 693
I20260808 02:15:27.310665 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.184.148:41519/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:32963
--local_ip_for_outbound_sockets=127.18.184.131
--tserver_master_addrs=127.18.184.190:34717
--webserver_port=39263
--webserver_interface=127.18.184.131
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.184.148:41519/kms with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:15:27 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:27.328284 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.309571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:54941 (local address 127.18.184.190:34717)
0808 02:15:27.309729 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:27.309733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:27.313646 (+ 3913us) server_negotiation.cc:407] Connection header received
0808 02:15:27.314924 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:27.314928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:27.314995 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:27.315055 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:27.316877 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.317811 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.319610 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.319985 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.320067 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:27.323494 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:27.323499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:27.323502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:27.323536 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:27.325461 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:27.326152 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:27.326155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:27.326157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:27.326253 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:27.326734 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:27.326738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:27.326739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:27.326997 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:27.327086 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:27.327928 (+ 842us) server_negotiation.cc:299] Negotiation successful
0808 02:15:27.328097 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:15:27.330551 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.310140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34717 (local address 127.18.184.130:54941)
0808 02:15:27.313377 (+ 3237us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.313447 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:27.314602 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:27.315543 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:27.315556 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:27.315965 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:27.316679 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:27.316701 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.317982 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.317986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:27.319444 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:27.319456 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.320293 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.320297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:27.320495 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:27.321276 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:27.321298 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:27.323260 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:27.325607 (+ 2347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:27.325614 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:27.325646 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:27.326021 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:27.326385 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:27.326390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:27.326391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:27.326596 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:27.327180 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:27.327188 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:27.327581 (+ 393us) client_negotiation.cc:769] Sending connection context
0808 02:15:27.328850 (+ 1269us) client_negotiation.cc:240] Negotiation successful
0808 02:15:27.329259 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2946,"thread_start_us":148,"threads_started":1}
I20260808 02:15:27.332355 833 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34717
I20260808 02:15:27.332644 833 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:27.333317 833 heartbeater.cc:511] Master 127.18.184.190:34717 requested a full tablet report, sending...
W20260808 02:15:27.335166 833 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:34717 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 02:15:27.335307 833 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:27.335525 833 heartbeater.cc:511] Master 127.18.184.190:34717 requested a full tablet report, sending...
I20260808 02:15:27.336423 833 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:27.336611 833 heartbeater.cc:511] Master 127.18.184.190:34717 requested a full tablet report, sending...
W20260808 02:15:27.337471 833 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:15:27.455225 836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:27.455689 836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:27.455778 836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:27.462293 836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:27.462393 836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:27.462483 836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:15:27.470947 836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.184.148:41519/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:32963
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=39263
--tserver_master_addrs=127.18.184.190:34717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:27.472473 836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:27.473939 836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:27.486238 841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:27.486227 845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:27.486714 843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:27.486796 836 server_base.cc:1034] running on GCE node
I20260808 02:15:27.487486 836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:27.488314 836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:27.491647 836 hybrid_clock.cc:648] HybridClock initialized: now 1786155327491600 us; error 48 us; skew 500 ppm
Aug 08 02:15:27 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:27.495018 836 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:15:27.496104 836 webserver.cc:460] Webserver started at http://127.18.184.131:39263/ using document root <none> and password file <none>
I20260808 02:15:27.496685 836 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:27.496767 836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:15:27 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:15:27.523063 836 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.003s sys 0.000s
I20260808 02:15:27.530030 854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.531755 836 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260808 02:15:27.531898 836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "cdc717c386254cbbb3afa9025b523397"
format_stamp: "Formatted at 2026-08-08 02:15:26 on dist-test-slave-9gcw"
server_key: "ba752616ba2dcb39c7c7e28edfb2d59e"
server_key_iv: "48857c7c771e93a5ba9f6cb2c580be61"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:27.532675 836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:27.554898 836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:27.558626 836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:27.558857 836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:27.559495 836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:27.560513 836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:27.560578 836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.560632 836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:27.560660 836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.571776 836 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:32963
I20260808 02:15:27.573202 836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:15:27.574074 979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:32963 every 8 connection(s)
I20260808 02:15:27.578989 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 836
Aug 08 02:15:27 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:27.590546 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.579712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57658 (local address 127.18.184.190:34717)
0808 02:15:27.580553 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:27.580561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:27.580573 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:15:27.580606 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:27.580608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:27.580648 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:27.580726 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:27.581802 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.582599 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.583814 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.584175 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.584292 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:27.586734 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:27.586738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:27.586740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:27.586792 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:27.588578 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:27.589125 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:27.589129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:27.589131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:27.589285 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:27.589584 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:27.589589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:27.589591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:27.589794 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:27.590049 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:27.590170 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 02:15:27.590310 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":124,"threads_started":1}
I20260808 02:15:27.591703 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.575772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:56389 (local address 127.18.184.190:34717)
0808 02:15:27.575915 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:27.575923 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:27.577131 (+ 1208us) server_negotiation.cc:407] Connection header received
0808 02:15:27.578392 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:27.578396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:27.578460 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:27.578571 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:27.580070 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.580875 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.582649 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.582974 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.583066 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:27.586161 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:27.586166 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:27.586170 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:27.586197 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:27.588287 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:27.589206 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:27.589211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:27.589213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:27.589310 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:27.589786 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:27.589789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:27.589791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:27.590008 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:27.590096 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:27.591131 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0808 02:15:27.591556 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:15:27.592751 985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.576263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34717 (local address 127.18.184.131:56389)
0808 02:15:27.576958 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.577021 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:27.578163 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:27.578766 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:27.578777 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:27.579194 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:27.579909 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:27.579929 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.581028 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.581032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:27.582512 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:27.582523 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:27.583239 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:27.583243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:27.583441 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:27.584145 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:27.584168 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:27.585945 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:27.588484 (+ 2539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:27.588491 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:27.588514 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:27.589029 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:27.589436 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:27.589440 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:27.589442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:27.589664 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:27.590157 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:27.590163 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:27.590542 (+ 379us) client_negotiation.cc:769] Sending connection context
0808 02:15:27.591081 (+ 539us) client_negotiation.cc:240] Negotiation successful
0808 02:15:27.591455 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":189,"threads_started":1}
I20260808 02:15:27.593983 980 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34717
I20260808 02:15:27.594255 980 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:27.594864 980 heartbeater.cc:511] Master 127.18.184.190:34717 requested a full tablet report, sending...
I20260808 02:15:27.596422 477 ts_manager.cc:194] Registered new tserver with Master: cdc717c386254cbbb3afa9025b523397 (127.18.184.131:32963)
I20260808 02:15:27.597666 477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:56389
Aug 08 02:15:27 dist-test-slave-9gcw krb5kdc[31872](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:28.060858 689 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:28.061275 689 heartbeater.cc:511] Master 127.18.184.190:34717 requested a full tablet report, sending...
I20260808 02:15:28.062158 477 ts_manager.cc:194] Registered new tserver with Master: 9606e43bc30248cd95f57b710b011f44 (127.18.184.129:45077)
I20260808 02:15:28.062940 477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:56169
I20260808 02:15:28.337700 833 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:28.338097 833 heartbeater.cc:511] Master 127.18.184.190:34717 requested a full tablet report, sending...
I20260808 02:15:28.338919 477 ts_manager.cc:194] Registered new tserver with Master: d2fbde563b864f92b85c575197887bf2 (127.18.184.130:46219)
I20260808 02:15:28.339720 477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:54941
I20260808 02:15:28.346539 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:15:28 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:15:28.360368 1001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.348249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57666 (local address 127.18.184.190:34717)
0808 02:15:28.348528 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:28.348534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:28.348884 (+ 350us) server_negotiation.cc:407] Connection header received
0808 02:15:28.349052 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:28.349056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:28.349113 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:28.349205 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:28.350166 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.350797 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.351671 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.351959 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.352061 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.353712 (+ 1651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:28.353716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:28.353719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:28.353746 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:28.358084 (+ 4338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:28.358931 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:28.358934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:28.358936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:28.359036 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:28.359380 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:28.359383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:28.359385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:28.359588 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:28.359738 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:28.360007 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 02:15:28.360149 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":121,"threads_started":1}
I20260808 02:15:28.364603 477 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:15:28.369946 477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:15:28.411468 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.400901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:45077 (local address 127.0.0.1:55156)
0808 02:15:28.403813 (+ 2912us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.403869 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:28.404000 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:28.404750 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:28.404757 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:28.404780 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:28.404921 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.404927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.407272 (+ 2345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.407276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.408694 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.408705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.410870 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.410874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.411004 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.411049 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 02:15:28.411156 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 02:15:28.411296 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2737,"thread_start_us":165,"threads_started":1}
I20260808 02:15:28.412101 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.402186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55156 (local address 127.18.184.129:45077)
0808 02:15:28.403374 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:28.403381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:28.404138 (+ 757us) server_negotiation.cc:407] Connection header received
0808 02:15:28.404218 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:28.404223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:28.404445 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:28.404580 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:28.405048 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.407125 (+ 2077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.408842 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.410533 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.410656 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.410809 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:28.411842 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0808 02:15:28.411955 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1073,"thread_start_us":127,"threads_started":1}
I20260808 02:15:28.415517 615 tablet_service.cc:1467] Processing CreateTablet for tablet 3013190c4ea34405af89964b7d443bc1 (DEFAULT_TABLE table=test-table [id=bfcc57ec4f6040038c5d546fa6fb06d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:28.416836 1010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.401748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:32963 (local address 127.0.0.1:38242)
0808 02:15:28.410527 (+ 8779us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.410542 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:28.410631 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:28.411321 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:28.411323 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:28.411340 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:28.411420 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.411425 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.414037 (+ 2612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.414039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.415144 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.415154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.416436 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.416439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.416523 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.416546 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:15:28.416614 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:15:28.416679 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8708,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260808 02:15:28.417174 615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3013190c4ea34405af89964b7d443bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:28.417347 1012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.401933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.18.184.131:32963)
0808 02:15:28.403558 (+ 1625us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:28.403565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:28.410736 (+ 7171us) server_negotiation.cc:407] Connection header received
0808 02:15:28.410839 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:28.410844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:28.411031 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:28.411174 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:28.411548 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.412693 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.415285 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.416130 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.416244 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.416372 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:28.417073 (+ 701us) server_negotiation.cc:299] Negotiation successful
0808 02:15:28.417204 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1523,"thread_start_us":80,"threads_started":1}
I20260808 02:15:28.417606 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.401390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:46219 (local address 127.0.0.1:52168)
0808 02:15:28.409520 (+ 8130us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.409536 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:28.409641 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:28.412838 (+ 3197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:28.412841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:28.412860 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:28.412953 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.412959 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.414008 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.414014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.416083 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.416096 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.417217 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.417220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.417314 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.417337 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:15:28.417404 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 02:15:28.417474 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8064,"mutex_wait_us":845,"thread_start_us":78,"threads_started":1}
I20260808 02:15:28.418227 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.401852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52168 (local address 127.18.184.130:46219)
0808 02:15:28.409363 (+ 7511us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:28.409371 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:28.409764 (+ 393us) server_negotiation.cc:407] Connection header received
0808 02:15:28.409862 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:28.409868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:28.410107 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:28.410263 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:28.413096 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.413884 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.416239 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.417064 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.417822 (+ 758us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.417948 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:28.417994 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 02:15:28.418080 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7385,"thread_start_us":87,"threads_started":1}
I20260808 02:15:28.420375 908 tablet_service.cc:1467] Processing CreateTablet for tablet 3013190c4ea34405af89964b7d443bc1 (DEFAULT_TABLE table=test-table [id=bfcc57ec4f6040038c5d546fa6fb06d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:28.420974 758 tablet_service.cc:1467] Processing CreateTablet for tablet 3013190c4ea34405af89964b7d443bc1 (DEFAULT_TABLE table=test-table [id=bfcc57ec4f6040038c5d546fa6fb06d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:28.421622 908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3013190c4ea34405af89964b7d443bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:28.421967 758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3013190c4ea34405af89964b7d443bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:28.432222 1014 tablet_bootstrap.cc:492] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2: Bootstrap starting.
I20260808 02:15:28.436003 1014 tablet_bootstrap.cc:654] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:28.436956 1014 log.cc:824] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:28.439666 1015 tablet_bootstrap.cc:492] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44: Bootstrap starting.
I20260808 02:15:28.439879 1014 tablet_bootstrap.cc:492] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2: No bootstrap required, opened a new log
I20260808 02:15:28.439997 1018 tablet_bootstrap.cc:492] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397: Bootstrap starting.
I20260808 02:15:28.440090 1014 ts_tablet_manager.cc:1397] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 02:15:28.442358 1018 tablet_bootstrap.cc:654] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:28.442561 1015 tablet_bootstrap.cc:654] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:28.443207 1018 log.cc:824] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:28.443522 1015 log.cc:824] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:28.447363 1018 tablet_bootstrap.cc:492] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397: No bootstrap required, opened a new log
I20260808 02:15:28.447643 1018 ts_tablet_manager.cc:1397] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 02:15:28.449558 1014 raft_consensus.cc:348] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9606e43bc30248cd95f57b710b011f44" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 45077 } } peers { permanent_uuid: "d2fbde563b864f92b85c575197887bf2" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46219 } } peers { permanent_uuid: "cdc717c386254cbbb3afa9025b523397" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 32963 } }
I20260808 02:15:28.449863 1014 raft_consensus.cc:374] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:28.449918 1014 raft_consensus.cc:729] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2fbde563b864f92b85c575197887bf2, State: Initialized, Role: FOLLOWER
I20260808 02:15:28.452672 1018 raft_consensus.cc:348] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9606e43bc30248cd95f57b710b011f44" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 45077 } } peers { permanent_uuid: "d2fbde563b864f92b85c575197887bf2" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46219 } } peers { permanent_uuid: "cdc717c386254cbbb3afa9025b523397" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 32963 } }
I20260808 02:15:28.453071 1018 raft_consensus.cc:374] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:28.453210 1018 raft_consensus.cc:729] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdc717c386254cbbb3afa9025b523397, State: Initialized, Role: FOLLOWER
I20260808 02:15:28.453835 1018 consensus_queue.cc:260] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9606e43bc30248cd95f57b710b011f44" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 45077 } } peers { permanent_uuid: "d2fbde563b864f92b85c575197887bf2" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46219 } } peers { permanent_uuid: "cdc717c386254cbbb3afa9025b523397" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 32963 } }
I20260808 02:15:28.455184 1018 ts_tablet_manager.cc:1428] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260808 02:15:28.455667 980 heartbeater.cc:503] Master 127.18.184.190:34717 was elected leader, sending a full tablet report...
I20260808 02:15:28.456713 1014 consensus_queue.cc:260] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9606e43bc30248cd95f57b710b011f44" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 45077 } } peers { permanent_uuid: "d2fbde563b864f92b85c575197887bf2" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46219 } } peers { permanent_uuid: "cdc717c386254cbbb3afa9025b523397" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 32963 } }
I20260808 02:15:28.457583 833 heartbeater.cc:503] Master 127.18.184.190:34717 was elected leader, sending a full tablet report...
I20260808 02:15:28.457933 1014 ts_tablet_manager.cc:1428] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2: Time spent starting tablet: real 0.018s user 0.008s sys 0.000s
I20260808 02:15:28.468046 1015 tablet_bootstrap.cc:492] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44: No bootstrap required, opened a new log
I20260808 02:15:28.468312 1015 ts_tablet_manager.cc:1397] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.000s
I20260808 02:15:28.472896 1015 raft_consensus.cc:348] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9606e43bc30248cd95f57b710b011f44" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 45077 } } peers { permanent_uuid: "d2fbde563b864f92b85c575197887bf2" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46219 } } peers { permanent_uuid: "cdc717c386254cbbb3afa9025b523397" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 32963 } }
I20260808 02:15:28.473479 1015 raft_consensus.cc:374] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:28.473592 1015 raft_consensus.cc:729] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9606e43bc30248cd95f57b710b011f44, State: Initialized, Role: FOLLOWER
I20260808 02:15:28.474363 1015 consensus_queue.cc:260] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9606e43bc30248cd95f57b710b011f44" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 45077 } } peers { permanent_uuid: "d2fbde563b864f92b85c575197887bf2" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46219 } } peers { permanent_uuid: "cdc717c386254cbbb3afa9025b523397" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 32963 } }
I20260808 02:15:28.475663 1015 ts_tablet_manager.cc:1428] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:15:28.476140 689 heartbeater.cc:503] Master 127.18.184.190:34717 was elected leader, sending a full tablet report...
W20260808 02:15:28.530153 690 tablet.cc:2369] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:15:28.559432 834 tablet.cc:2369] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:15:28.576251 981 tablet.cc:2369] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 02:15:28 dist-test-slave-9gcw krb5kdc[31872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for HTTP/127.18.184.148@KRBTEST.COM
I20260808 02:15:28.706550 1023 raft_consensus.cc:482] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:28.706815 1023 raft_consensus.cc:504] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9606e43bc30248cd95f57b710b011f44" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 45077 } } peers { permanent_uuid: "d2fbde563b864f92b85c575197887bf2" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46219 } } peers { permanent_uuid: "cdc717c386254cbbb3afa9025b523397" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 32963 } }
I20260808 02:15:28.708056 1023 leader_election.cc:288] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2fbde563b864f92b85c575197887bf2 (127.18.184.130:46219), cdc717c386254cbbb3afa9025b523397 (127.18.184.131:32963)
I20260808 02:15:28.713971 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.708695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:33987 (local address 127.18.184.130:46219)
0808 02:15:28.708862 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:28.708867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:28.709105 (+ 238us) server_negotiation.cc:407] Connection header received
0808 02:15:28.709372 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:28.709378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:28.709435 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:28.709488 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:28.709995 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.710759 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.712417 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.713007 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.713117 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.713207 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:28.713718 (+ 511us) server_negotiation.cc:299] Negotiation successful
0808 02:15:28.713813 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:15:28.714700 1027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.708256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:46219 (local address 127.18.184.129:33987)
0808 02:15:28.709006 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.709024 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:28.709226 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:28.709644 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:28.709647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:28.709668 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:28.709848 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.709857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.710930 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.710934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.712262 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.712274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.713318 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.713322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.713503 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.713532 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:15:28.714440 (+ 908us) client_negotiation.cc:240] Negotiation successful
0808 02:15:28.714535 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":113,"threads_started":1}
I20260808 02:15:28.715677 786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3013190c4ea34405af89964b7d443bc1" candidate_uuid: "9606e43bc30248cd95f57b710b011f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2fbde563b864f92b85c575197887bf2" is_pre_election: true
I20260808 02:15:28.716068 786 raft_consensus.cc:2454] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9606e43bc30248cd95f57b710b011f44 in term 0.
I20260808 02:15:28.716617 569 leader_election.cc:302] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9606e43bc30248cd95f57b710b011f44, d2fbde563b864f92b85c575197887bf2; no voters:
I20260808 02:15:28.717077 1023 raft_consensus.cc:2781] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:15:28.717209 1023 raft_consensus.cc:482] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:15:28.717265 1023 raft_consensus.cc:3037] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:28.718869 1023 raft_consensus.cc:504] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9606e43bc30248cd95f57b710b011f44" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 45077 } } peers { permanent_uuid: "d2fbde563b864f92b85c575197887bf2" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46219 } } peers { permanent_uuid: "cdc717c386254cbbb3afa9025b523397" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 32963 } }
I20260808 02:15:28.719293 1023 leader_election.cc:288] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [CANDIDATE]: Term 1 election: Requested vote from peers d2fbde563b864f92b85c575197887bf2 (127.18.184.130:46219), cdc717c386254cbbb3afa9025b523397 (127.18.184.131:32963)
I20260808 02:15:28.720059 786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3013190c4ea34405af89964b7d443bc1" candidate_uuid: "9606e43bc30248cd95f57b710b011f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2fbde563b864f92b85c575197887bf2"
I20260808 02:15:28.720203 786 raft_consensus.cc:3037] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:28.721761 786 raft_consensus.cc:2454] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9606e43bc30248cd95f57b710b011f44 in term 1.
I20260808 02:15:28.722200 569 leader_election.cc:302] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9606e43bc30248cd95f57b710b011f44, d2fbde563b864f92b85c575197887bf2; no voters:
I20260808 02:15:28.722474 1023 raft_consensus.cc:2781] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:28.722721 1023 raft_consensus.cc:686] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [term 1 LEADER]: Becoming Leader. State: Replica: 9606e43bc30248cd95f57b710b011f44, State: Running, Role: LEADER
I20260808 02:15:28.723131 1023 consensus_queue.cc:237] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9606e43bc30248cd95f57b710b011f44" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 45077 } } peers { permanent_uuid: "d2fbde563b864f92b85c575197887bf2" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46219 } } peers { permanent_uuid: "cdc717c386254cbbb3afa9025b523397" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 32963 } }
I20260808 02:15:28.727290 477 catalog_manager.cc:5495] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9606e43bc30248cd95f57b710b011f44 (127.18.184.129). New cstate: current_term: 1 leader_uuid: "9606e43bc30248cd95f57b710b011f44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9606e43bc30248cd95f57b710b011f44" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2fbde563b864f92b85c575197887bf2" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdc717c386254cbbb3afa9025b523397" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 32963 } health_report { overall_health: UNKNOWN } } }
I20260808 02:15:28.729463 1012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.708726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:48631 (local address 127.18.184.131:32963)
0808 02:15:28.708918 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:28.708923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:28.715529 (+ 6606us) server_negotiation.cc:407] Connection header received
0808 02:15:28.715763 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:28.715767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:28.715829 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:28.715902 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:28.725774 (+ 9872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.726501 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.728118 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.728674 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.728779 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.728848 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:28.729233 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 02:15:28.729328 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260808 02:15:28.730131 1027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.708363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:32963 (local address 127.18.184.129:48631)
0808 02:15:28.715436 (+ 7073us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.715453 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:28.715620 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:28.716134 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:28.716137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:28.716233 (+ 96us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:28.716320 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.716328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.726659 (+ 10331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.726662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.727952 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.727964 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.728954 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.728958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.729050 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.729076 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:15:28.729883 (+ 807us) client_negotiation.cc:240] Negotiation successful
0808 02:15:28.729979 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7005,"thread_start_us":102,"threads_started":1}
I20260808 02:15:28.730996 931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3013190c4ea34405af89964b7d443bc1" candidate_uuid: "9606e43bc30248cd95f57b710b011f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdc717c386254cbbb3afa9025b523397" is_pre_election: true
I20260808 02:15:28.731319 931 raft_consensus.cc:2454] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9606e43bc30248cd95f57b710b011f44 in term 0.
I20260808 02:15:28.731650 930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3013190c4ea34405af89964b7d443bc1" candidate_uuid: "9606e43bc30248cd95f57b710b011f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdc717c386254cbbb3afa9025b523397"
I20260808 02:15:28.731781 930 raft_consensus.cc:3037] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:28.733208 930 raft_consensus.cc:2454] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9606e43bc30248cd95f57b710b011f44 in term 1.
I20260808 02:15:28.782963 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.778138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55158 (local address 127.18.184.129:45077)
0808 02:15:28.778281 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:28.778286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:28.778301 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:15:28.778436 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:28.778440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:28.778510 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:28.778636 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:15:28.778927 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.779728 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.781003 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.781362 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.781469 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.781890 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:15:28.782480 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:15:28.782613 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:28.782728 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 02:15:28.782809 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:15:28.790889 930 raft_consensus.cc:1261] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397 [term 1 FOLLOWER]: Refusing update from remote peer 9606e43bc30248cd95f57b710b011f44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:28.792176 1028 consensus_queue.cc:1035] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdc717c386254cbbb3afa9025b523397" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 32963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:28.798347 786 raft_consensus.cc:1261] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2 [term 1 FOLLOWER]: Refusing update from remote peer 9606e43bc30248cd95f57b710b011f44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:28.799096 1028 consensus_queue.cc:1035] T 3013190c4ea34405af89964b7d443bc1 P 9606e43bc30248cd95f57b710b011f44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2fbde563b864f92b85c575197887bf2" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:28.807901 1031 mvcc.cc:205] Tried to move back new op lower bound from 7316092226718134272 to 7316092226455552000. Current Snapshot: MvccSnapshot[applied={T|T < 7316092226718134272}]
I20260808 02:15:28.808480 1032 mvcc.cc:205] Tried to move back new op lower bound from 7316092226718134272 to 7316092226455552000. Current Snapshot: MvccSnapshot[applied={T|T < 7316092226718134272}]
I20260808 02:15:28.812206 1035 mvcc.cc:205] Tried to move back new op lower bound from 7316092226718134272 to 7316092226455552000. Current Snapshot: MvccSnapshot[applied={T|T < 7316092226718134272}]
I20260808 02:15:28.822503 477 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:28.822717 477 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:28.826040 477 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97df1e753be44a038cda01d1a0b9642a: Sending DeleteTablet for 3 replicas of tablet 3013190c4ea34405af89964b7d443bc1
I20260808 02:15:28.827097 615 tablet_service.cc:1514] Processing DeleteTablet for tablet 3013190c4ea34405af89964b7d443bc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:28 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:55156
I20260808 02:15:28.828325 758 tablet_service.cc:1514] Processing DeleteTablet for tablet 3013190c4ea34405af89964b7d443bc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:28 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:52168
I20260808 02:15:28.829399 908 tablet_service.cc:1514] Processing DeleteTablet for tablet 3013190c4ea34405af89964b7d443bc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:28 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:38242
I20260808 02:15:28.831050 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 518
I20260808 02:15:28.841598 1042 tablet_replica.cc:330] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2: stopping tablet replica
I20260808 02:15:28.842059 1042 raft_consensus.cc:2229] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:28.842487 1042 raft_consensus.cc:2258] T 3013190c4ea34405af89964b7d443bc1 P d2fbde563b864f92b85c575197887bf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:28.842921 1043 tablet_replica.cc:330] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397: stopping tablet replica
I20260808 02:15:28.843250 1043 raft_consensus.cc:2229] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:28.843554 1043 raft_consensus.cc:2258] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 02:15:28.845286 460 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:45077 (error 108)
I20260808 02:15:28.845448 1043 ts_tablet_manager.cc:1905] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:28.846283 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 693
I20260808 02:15:28.849054 1043 ts_tablet_manager.cc:1918] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:28.849176 1043 log.cc:1196] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/3013190c4ea34405af89964b7d443bc1
I20260808 02:15:28.849530 1043 ts_tablet_manager.cc:1939] T 3013190c4ea34405af89964b7d443bc1 P cdc717c386254cbbb3afa9025b523397: Deleting consensus metadata
I20260808 02:15:28.850956 460 catalog_manager.cc:4841] TS cdc717c386254cbbb3afa9025b523397 (127.18.184.131:32963): tablet 3013190c4ea34405af89964b7d443bc1 (table test-table [id=bfcc57ec4f6040038c5d546fa6fb06d3]) successfully deleted
I20260808 02:15:28.859027 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.858564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:45077 (local address 127.0.0.1:55172)
0808 02:15:28.858805 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.858894 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:45077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260808 02:15:28.859306 460 catalog_manager.cc:4584] TS 9606e43bc30248cd95f57b710b011f44 (127.18.184.129:45077): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3013190c4ea34405af89964b7d443bc1: Network error: Client connection negotiation failed: client connection to 127.18.184.129:45077: connect: Connection refused (error 111)
I20260808 02:15:28.860532 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 836
I20260808 02:15:28.860880 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.860551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:46219 (local address 127.0.0.1:52178)
0808 02:15:28.860743 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.860799 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.130:46219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260808 02:15:28.872246 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 441
I20260808 02:15:28.886216 19170 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 02:15:29.231760 19170 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 02:15:29.231912 19170 mini_ranger.cc:66] Stopping Ranger...
I20260808 02:15:29.588045 19170 mini_ranger.cc:68] Stopped Ranger
2026-08-08 02:15:29.588 UTC [31903] LOG: received smart shutdown request
2026-08-08 02:15:29.594 UTC [31903] LOG: background worker "logical replication launcher" (PID 31912) exited with exit code 1
2026-08-08 02:15:29.594 UTC [31907] LOG: shutting down
2026-08-08 02:15:29.804 UTC [31903] LOG: database system is shut down
2026-08-08T02:15:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51924 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:15:30 dist-test-slave-9gcw krb5kdc[1055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:15:30 dist-test-slave-9gcw krb5kdc[1055](info): set up 2 sockets
Aug 08 02:15:30 dist-test-slave-9gcw krb5kdc[1055](info): commencing operation
krb5kdc: starting...
W20260808 02:15:31.059337 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:15:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:15:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:15:31.196468 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:41605
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:41605,127.18.184.189:38737,127.18.184.188:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:31.313601 1069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:31.313889 1069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:31.313933 1069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:31.317973 1069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:31.318037 1069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:31.318056 1069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:31.318073 1069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:31.318089 1069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:31.323539 1069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:41605,127.18.184.189:38737,127.18.184.188:40807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:41605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:31.324636 1069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:31.325673 1069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:31.335492 1077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:31.335492 1075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:31.335492 1074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:31.335740 1069 server_base.cc:1034] running on GCE node
I20260808 02:15:31.336347 1069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:31.337684 1069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:31.339033 1069 hybrid_clock.cc:648] HybridClock initialized: now 1786155331338994 us; error 58 us; skew 500 ppm
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:31.342386 1069 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:15:31.343326 1069 webserver.cc:460] Webserver started at http://127.18.184.190:39083/ using document root <none> and password file <none>
I20260808 02:15:31.343887 1069 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:31.343948 1069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:31.344121 1069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:31.346293 1069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6"
format_stamp: "Formatted at 2026-08-08 02:15:31 on dist-test-slave-9gcw"
server_key: "4e2a95b53610514e46d70ebdcf059373"
server_key_iv: "74aa0cd108fd898b1a5dff3858946895"
server_key_version: "encryptionkey@0"
I20260808 02:15:31.346833 1069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6"
format_stamp: "Formatted at 2026-08-08 02:15:31 on dist-test-slave-9gcw"
server_key: "4e2a95b53610514e46d70ebdcf059373"
server_key_iv: "74aa0cd108fd898b1a5dff3858946895"
server_key_version: "encryptionkey@0"
I20260808 02:15:31.350679 1069 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:15:31.353091 1085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:31.354413 1069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:15:31.354549 1069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6"
format_stamp: "Formatted at 2026-08-08 02:15:31 on dist-test-slave-9gcw"
server_key: "4e2a95b53610514e46d70ebdcf059373"
server_key_iv: "74aa0cd108fd898b1a5dff3858946895"
server_key_version: "encryptionkey@0"
I20260808 02:15:31.354660 1069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:31.369619 1069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:31.372476 1069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:31.372691 1069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:31.380324 1069 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:41605
I20260808 02:15:31.380630 1139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:41605 every 8 connection(s)
I20260808 02:15:31.381448 1069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:31.384416 1140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:31.384693 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 1069
I20260808 02:15:31.384882 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:15:31.385351 19170 external_mini_cluster.cc:1428] Setting key 6400bf9f1c3a7b646cfd2497e52fb959
I20260808 02:15:31.388475 1140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.184.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.189@KRBTEST.COM" created.
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:31.405711 1144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.390963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48908 (local address 127.18.184.190:41605)
0808 02:15:31.391699 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.391718 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.391780 (+ 62us) server_negotiation.cc:407] Connection header received
0808 02:15:31.392800 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.392810 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.393095 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.393377 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.394239 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.395875 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.397055 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.397792 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.397941 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.400700 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.400706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.400717 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.400737 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.402431 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.402934 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.402940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.402941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.403050 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.403376 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.403379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.403380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.403765 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.403902 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.404282 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.404582 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":555,"thread_start_us":188,"threads_started":1}
I20260808 02:15:31.405711 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.390963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:41605 (local address 127.0.0.1:48908)
0808 02:15:31.391468 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.391531 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:31.392413 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:31.393366 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:31.393371 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:31.393772 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:31.394049 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.394063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.396030 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.396033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.396900 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.396910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.397962 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.397965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.398060 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.398693 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:31.398710 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:31.400540 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:31.402562 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.402572 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.402580 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.402820 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.403149 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.403152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.403154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.403273 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.403886 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:31.403892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:31.404036 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 02:15:31.404210 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 02:15:31.404635 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":147,"thread_start_us":134,"threads_started":1}
I20260808 02:15:31.408488 1140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:15:31.409356 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.409001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:38737 (local address 127.0.0.1:46242)
0808 02:15:31.409183 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.409287 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 02:15:31.409564 1088 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
I20260808 02:15:31.410430 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.410183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:38737 (local address 127.0.0.1:46256)
0808 02:15:31.410325 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.410377 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260808 02:15:31.410647 1140 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.189:38737: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.189.keytab.
Entry for principal kudu/127.18.184.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.189.keytab.
I20260808 02:15:31.417902 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.189:38737
--webserver_interface=127.18.184.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.189.keytab
--principal=kudu/127.18.184.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:41605,127.18.184.189:38737,127.18.184.188:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:31.459915 1140 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } attempt: 1
I20260808 02:15:31.460917 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.460554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:38737 (local address 127.0.0.1:46268)
0808 02:15:31.460728 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.460827 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 02:15:31.461728 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.461494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:38737 (local address 127.0.0.1:46284)
0808 02:15:31.461613 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.461668 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 02:15:31.461869 1140 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.189:38737: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
I20260808 02:15:31.530055 1140 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } attempt: 2
I20260808 02:15:31.531085 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.530685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:38737 (local address 127.0.0.1:46288)
0808 02:15:31.530879 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.530974 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260808 02:15:31.531945 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.531740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:38737 (local address 127.0.0.1:46294)
0808 02:15:31.531848 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.531893 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 02:15:31.532065 1140 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.189:38737: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
W20260808 02:15:31.545295 1147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:31.545600 1147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:31.545712 1147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:31.549922 1147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:31.550009 1147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:31.550055 1147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:31.550113 1147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:31.550153 1147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:31.555785 1147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:41605,127.18.184.189:38737,127.18.184.188:40807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.189.keytab
--principal=kudu/127.18.184.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.189:38737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.184.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:31.556998 1147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:31.558226 1147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:31.567003 1154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:31.567070 1152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:31.567234 1147 server_base.cc:1034] running on GCE node
W20260808 02:15:31.567031 1156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:31.567890 1147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:31.569029 1147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:31.570222 1147 hybrid_clock.cc:648] HybridClock initialized: now 1786155331570189 us; error 46 us; skew 500 ppm
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:31.573218 1147 init.cc:377] Logged in from keytab as kudu/127.18.184.189@KRBTEST.COM (short username kudu)
I20260808 02:15:31.574105 1147 webserver.cc:460] Webserver started at http://127.18.184.189:36081/ using document root <none> and password file <none>
I20260808 02:15:31.574669 1147 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:31.574764 1147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:31.575073 1147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:31.577224 1147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/data/instance:
uuid: "27e9482f10c04abfa200356834980a64"
format_stamp: "Formatted at 2026-08-08 02:15:31 on dist-test-slave-9gcw"
server_key: "1fc6e5c46c91f45c3ad0a655761f32e5"
server_key_iv: "4aa814f5941f6700872f6516aca37ea0"
server_key_version: "encryptionkey@0"
I20260808 02:15:31.577811 1147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/wal/instance:
uuid: "27e9482f10c04abfa200356834980a64"
format_stamp: "Formatted at 2026-08-08 02:15:31 on dist-test-slave-9gcw"
server_key: "1fc6e5c46c91f45c3ad0a655761f32e5"
server_key_iv: "4aa814f5941f6700872f6516aca37ea0"
server_key_version: "encryptionkey@0"
I20260808 02:15:31.581820 1147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 02:15:31.584378 1166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:31.585711 1147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 02:15:31.585886 1147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/wal
uuid: "27e9482f10c04abfa200356834980a64"
format_stamp: "Formatted at 2026-08-08 02:15:31 on dist-test-slave-9gcw"
server_key: "1fc6e5c46c91f45c3ad0a655761f32e5"
server_key_iv: "4aa814f5941f6700872f6516aca37ea0"
server_key_version: "encryptionkey@0"
I20260808 02:15:31.586004 1147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:31.618875 1147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:31.622805 1147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:31.623044 1147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:31.623210 1140 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } attempt: 3
I20260808 02:15:31.624058 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.623715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:38737 (local address 127.0.0.1:46302)
0808 02:15:31.623876 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.623949 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 02:15:31.624996 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.624687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:38737 (local address 127.0.0.1:46318)
0808 02:15:31.624821 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.624894 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 02:15:31.625170 1140 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.189:38737: Network error: Client connection negotiation failed: client connection to 127.18.184.189:38737: connect: Connection refused (error 111)
I20260808 02:15:31.631104 1147 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.189:38737
I20260808 02:15:31.631405 1222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.189:38737 every 8 connection(s)
I20260808 02:15:31.632131 1147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
I20260808 02:15:31.635154 1223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:31.636529 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 1147
I20260808 02:15:31.636669 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-1/wal/instance
I20260808 02:15:31.636981 19170 external_mini_cluster.cc:1428] Setting key 35eccfee46bbde7610fa8c7f5c3518cf
I20260808 02:15:31.639165 1223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.184.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.188@KRBTEST.COM" created.
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.189@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:31.654000 1144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.641137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.18.184.190:41605)
0808 02:15:31.641322 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.641330 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.642188 (+ 858us) server_negotiation.cc:407] Connection header received
0808 02:15:31.643151 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.643157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.643231 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.643351 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.644512 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.645413 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.646639 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.646988 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.647081 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.650109 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.650113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.650116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.650153 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.651683 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.652259 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.652263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.652265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.652360 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.652734 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.652740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.652743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.652955 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.653068 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.653684 (+ 616us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.653821 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 02:15:31.654888 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.641518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:41605 (local address 127.0.0.1:48916)
0808 02:15:31.642010 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.642063 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:31.642933 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:31.643503 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:31.643510 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:31.643853 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:31.644346 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.644359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.645548 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.645551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.646516 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.646526 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.647195 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.647199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.647367 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.647980 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:31.648008 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:31.649920 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:31.651825 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.651831 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.651848 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.652129 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.652469 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.652474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.652476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.652631 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.653068 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:31.653077 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:31.653428 (+ 351us) client_negotiation.cc:769] Sending connection context
0808 02:15:31.653628 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 02:15:31.653985 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":145,"threads_started":1}
I20260808 02:15:31.656481 1223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.189@KRBTEST.COM for kudu/127.18.184.189@KRBTEST.COM
I20260808 02:15:31.667179 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.657074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:38737 (local address 127.0.0.1:46322)
0808 02:15:31.657224 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.657238 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:31.657315 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:31.657990 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:31.657993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:31.658180 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:31.658293 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.658299 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.659377 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.659384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.659976 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.659985 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.660657 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.660661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.660754 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.661354 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:31.661366 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:31.663418 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:31.665499 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.665502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.665503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.665696 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.666017 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.666019 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.666020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.666100 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.666614 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:31.666617 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:31.666701 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 02:15:31.666810 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 02:15:31.667006 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 02:15:31.667210 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.657248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46322 (local address 127.18.184.189:38737)
0808 02:15:31.657604 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.657612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.657630 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:15:31.657662 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.657666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.657840 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.658006 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.658405 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.659266 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.660100 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.660534 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.660657 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.663565 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.663575 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.663588 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.663613 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.665392 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.665803 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.665808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.665809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.665931 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.666184 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.666186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.666187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.666520 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.666635 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.666909 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.667055 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":171,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.188.keytab.
Entry for principal kudu/127.18.184.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.188.keytab.
I20260808 02:15:31.668697 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.188:40807
--webserver_interface=127.18.184.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.188.keytab
--principal=kudu/127.18.184.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:41605,127.18.184.189:38737,127.18.184.188:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:31.669605 1223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:15:31.670342 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.670071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51050)
0808 02:15:31.670192 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.670276 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 02:15:31.670593 1168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
I20260808 02:15:31.671455 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.671253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51066)
0808 02:15:31.671354 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.671405 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 02:15:31.671613 1223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:40807: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
I20260808 02:15:31.720943 1223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } attempt: 1
I20260808 02:15:31.722385 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.721788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51076)
0808 02:15:31.722185 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.722270 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 02:15:31.723338 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.723089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51090)
0808 02:15:31.723209 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.723268 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 02:15:31.723486 1223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:40807: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
I20260808 02:15:31.768355 1140 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } attempt: 4
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.189@KRBTEST.COM
I20260808 02:15:31.779737 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.769044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:38737 (local address 127.0.0.1:46324)
0808 02:15:31.769316 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.769332 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:31.769430 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:31.769735 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:31.769737 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:31.769911 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:31.770022 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.770028 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.771128 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.771131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.772105 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.772116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.773437 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.773440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.773551 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.774371 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:31.774387 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:31.775943 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:31.778221 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.778224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.778225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.778437 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.778751 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.778754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.778755 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.778845 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.779301 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:31.779303 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:31.779397 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 02:15:31.779480 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 02:15:31.779600 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260808 02:15:31.779860 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.769206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46324 (local address 127.18.184.189:38737)
0808 02:15:31.769361 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.769366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.769387 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:15:31.769537 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.769542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.769601 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.769691 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.770154 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.770977 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.772250 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.773264 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.773395 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.776137 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.776144 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.776147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.776178 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.778075 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.778542 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.778546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.778548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.778641 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.778941 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.778944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.778946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.779189 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.779308 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.779565 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.779703 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:15:31.780957 1140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:15:31.781862 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.781513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51104)
0808 02:15:31.781746 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.781799 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 02:15:31.782537 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.782340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51110)
0808 02:15:31.782441 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.782492 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 02:15:31.782686 1140 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:40807: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
I20260808 02:15:31.791656 1223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } attempt: 2
I20260808 02:15:31.792354 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.792091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51112)
0808 02:15:31.792223 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.792285 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 02:15:31.793100 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.792891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51114)
0808 02:15:31.792996 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.793046 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 02:15:31.793298 1223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:40807: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
W20260808 02:15:31.796840 1229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:31.797127 1229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:31.797253 1229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:31.801333 1229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:31.801410 1229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:31.801455 1229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:31.801491 1229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:31.801527 1229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:31.806993 1229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:41605,127.18.184.189:38737,127.18.184.188:40807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.188.keytab
--principal=kudu/127.18.184.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.188:40807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.184.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:31.808157 1229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:31.809279 1229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:31.818511 1236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:31.818511 1239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:31.818567 1237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:31.818964 1229 server_base.cc:1034] running on GCE node
I20260808 02:15:31.819581 1229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:31.820765 1229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:31.821945 1229 hybrid_clock.cc:648] HybridClock initialized: now 1786155331821922 us; error 31 us; skew 500 ppm
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:31.825264 1229 init.cc:377] Logged in from keytab as kudu/127.18.184.188@KRBTEST.COM (short username kudu)
I20260808 02:15:31.826368 1229 webserver.cc:460] Webserver started at http://127.18.184.188:34109/ using document root <none> and password file <none>
I20260808 02:15:31.827035 1229 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:31.827136 1229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:31.827380 1229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:31.829659 1229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/data/instance:
uuid: "19fe28805eb04cb7b82654eeb9d79099"
format_stamp: "Formatted at 2026-08-08 02:15:31 on dist-test-slave-9gcw"
server_key: "b8fab183f6702e822da45ef84f1414e7"
server_key_iv: "3d9abd48f69109e1f7f305ed9a07800a"
server_key_version: "encryptionkey@0"
I20260808 02:15:31.830286 1229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/wal/instance:
uuid: "19fe28805eb04cb7b82654eeb9d79099"
format_stamp: "Formatted at 2026-08-08 02:15:31 on dist-test-slave-9gcw"
server_key: "b8fab183f6702e822da45ef84f1414e7"
server_key_iv: "3d9abd48f69109e1f7f305ed9a07800a"
server_key_version: "encryptionkey@0"
I20260808 02:15:31.834367 1229 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:15:31.836979 1247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:31.838342 1229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:31.838498 1229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/wal
uuid: "19fe28805eb04cb7b82654eeb9d79099"
format_stamp: "Formatted at 2026-08-08 02:15:31 on dist-test-slave-9gcw"
server_key: "b8fab183f6702e822da45ef84f1414e7"
server_key_iv: "3d9abd48f69109e1f7f305ed9a07800a"
server_key_version: "encryptionkey@0"
I20260808 02:15:31.838615 1229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:31.841841 1140 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } attempt: 1
I20260808 02:15:31.842603 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.842322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51116)
0808 02:15:31.842464 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.842528 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 02:15:31.843390 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.843133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51118)
0808 02:15:31.843278 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.843330 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260808 02:15:31.843529 1140 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:40807: Network error: Client connection negotiation failed: client connection to 127.18.184.188:40807: connect: Connection refused (error 111)
I20260808 02:15:31.869804 1229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:31.873205 1229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:31.873445 1229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:31.881069 1229 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.188:40807
I20260808 02:15:31.881413 1304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.188:40807 every 8 connection(s)
I20260808 02:15:31.882172 1229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
I20260808 02:15:31.884459 1223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } attempt: 3
I20260808 02:15:31.885483 1306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:31.887485 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 1229
I20260808 02:15:31.887642 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155161841348-19170-0/minicluster-data/master-2/wal/instance
I20260808 02:15:31.887962 19170 external_mini_cluster.cc:1428] Setting key 92d09ba9dc5a04a8078e74d2653e3ecd
I20260808 02:15:31.890830 1306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.189@KRBTEST.COM for kudu/127.18.184.188@KRBTEST.COM
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.188@KRBTEST.COM
I20260808 02:15:31.900774 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.885021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51126)
0808 02:15:31.885229 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.885247 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:31.885355 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:31.887824 (+ 2469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:31.887828 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:31.888055 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:31.888205 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.888212 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.890775 (+ 2563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.890779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.891375 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.891390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.892311 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.892314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.892410 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.893036 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:31.893058 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:31.895030 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:31.898309 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.898315 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.898319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.898599 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.899065 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.899068 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.899070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.899177 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.900068 (+ 891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:31.900072 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:31.900210 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 02:15:31.900347 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 02:15:31.900548 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.188@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:31.902396 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.888665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51142 (local address 127.18.184.188:40807)
0808 02:15:31.888996 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.889005 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.889209 (+ 204us) server_negotiation.cc:407] Connection header received
0808 02:15:31.889368 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.889371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.889419 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.889511 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.890419 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.891188 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.892160 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.892508 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.892635 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.895642 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.895646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.895648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.895680 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.898382 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.899049 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.899054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.899059 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.899168 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.899571 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.899577 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.899578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.899925 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.900146 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.900579 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.900991 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":175,"server-negotiator.queue_time_us":250,"thread_start_us":133,"threads_started":1}
I20260808 02:15:31.903030 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.885868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.18.184.188:40807)
0808 02:15:31.886361 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.886378 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.886415 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:15:31.887073 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.887089 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.887434 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.888459 (+ 1025us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.888939 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.890636 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.891505 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.892176 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.892340 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.895206 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.895212 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.895221 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.895248 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.898181 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.898744 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.898751 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.898760 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.898952 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.899303 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.899309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.899312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.899914 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.900081 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.900549 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.902814 (+ 2265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":351,"spinlock_wait_cycles":18816,"thread_start_us":152,"threads_started":1}
I20260808 02:15:31.905678 1144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.893805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48924 (local address 127.18.184.190:41605)
0808 02:15:31.894470 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.894473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.894486 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:15:31.894556 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.894558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.894606 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.894690 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.895673 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.896507 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.897796 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.898239 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.898397 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.901615 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.901620 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.901623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.901660 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.903597 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.904146 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.904150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.904151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.904246 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.904564 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.904567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.904569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.904829 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.904931 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.905387 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.905515 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581}
I20260808 02:15:31.905705 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.893333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:41605 (local address 127.0.0.1:48924)
0808 02:15:31.893861 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.893921 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:31.894043 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:31.894789 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:31.894798 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:31.895301 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:31.895523 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.895539 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.896629 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.896632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.897585 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.897599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.898390 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.898394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.898532 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.899070 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:31.899121 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:31.901456 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:31.903726 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.903735 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.903742 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.904022 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.904366 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.904371 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.904373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.904461 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.904950 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:31.904956 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:31.905094 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 02:15:31.905296 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 02:15:31.905516 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"spinlock_wait_cycles":1280,"thread_start_us":112,"threads_started":1}
I20260808 02:15:31.907192 1306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:15:31.908142 1223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64: Bootstrap starting.
I20260808 02:15:31.910717 1140 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } attempt: 2
I20260808 02:15:31.911032 1223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:31.912092 1223 log.cc:824] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:31.915027 1223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64: No bootstrap required, opened a new log
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.188@KRBTEST.COM
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.188@KRBTEST.COM for kudu/127.18.184.189@KRBTEST.COM
I20260808 02:15:31.917842 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.905913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48936 (local address 127.18.184.190:41605)
0808 02:15:31.906308 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.906323 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.906655 (+ 332us) server_negotiation.cc:407] Connection header received
0808 02:15:31.906694 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.906697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.906742 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.906820 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.908304 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.908893 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.910045 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.910337 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.910423 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.913705 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.913709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.913712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.913739 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.915648 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.916186 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.916190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.916192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.916308 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.916638 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.916641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.916643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.916883 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.917110 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.917496 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.917623 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":81,"threads_started":1}
I20260808 02:15:31.920567 1223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } }
I20260808 02:15:31.920948 1223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:31.921077 1223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27e9482f10c04abfa200356834980a64, State: Initialized, Role: FOLLOWER
I20260808 02:15:31.921758 1223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } }
I20260808 02:15:31.922235 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.912670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46326 (local address 127.18.184.189:38737)
0808 02:15:31.912776 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.912780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.912794 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:15:31.912832 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.912834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.912876 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.912955 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.913554 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.914275 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.915464 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.915827 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.915928 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.918347 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.918351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.918353 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.918378 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.920119 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.920695 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.920699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.920700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.920784 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.921226 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.921233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.921234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.921507 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.921582 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.921968 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.922079 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 02:15:31.922605 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.911276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51146)
0808 02:15:31.911407 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.911425 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:31.911504 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:31.911939 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:31.911942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:31.912122 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:31.912239 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.912248 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.913240 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.913244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.914345 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.914356 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.915328 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.915332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.915432 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.915901 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:31.915916 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:31.918441 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:31.920902 (+ 2461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.920907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.920909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.921205 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.921531 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.921534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.921536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.921621 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.922091 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:31.922094 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:31.922208 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 02:15:31.922303 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 02:15:31.922453 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 02:15:31.922691 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.907757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:38737 (local address 127.0.0.1:46326)
0808 02:15:31.907924 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.907935 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:31.907995 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:31.913022 (+ 5027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:31.913027 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:31.913250 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:31.913397 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.913405 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.914418 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.914423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.915283 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.915295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.916063 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.916067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.916180 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.916614 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:31.916630 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:31.918226 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:31.920231 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.920236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.920239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.920586 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.920906 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.920909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.920911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.921010 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.921660 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:31.921666 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:31.921779 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 02:15:31.922372 (+ 593us) client_negotiation.cc:240] Negotiation successful
0808 02:15:31.922531 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"spinlock_wait_cycles":9984}
I20260808 02:15:31.923473 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.911551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51146 (local address 127.18.184.188:40807)
0808 02:15:31.911737 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.911740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.911753 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:15:31.911785 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.911787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.911824 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.911919 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.912369 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.913060 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.914483 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.915090 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.915193 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.918601 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.918605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.918608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.918904 (+ 296us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.920777 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.921325 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.921329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.921331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.921422 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.921735 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.921738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.921740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.921952 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.922041 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.923185 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.923318 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 02:15:31.923452 1306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:15:31.923996 1223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.189@KRBTEST.COM
I20260808 02:15:31.927086 1140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6: Bootstrap starting.
Aug 08 02:15:31 dist-test-slave-9gcw krb5kdc[1055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155331, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.188@KRBTEST.COM for kudu/127.18.184.188@KRBTEST.COM
I20260808 02:15:31.930068 1140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:31.931119 1140 log.cc:824] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:31.932605 1315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.919397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.18.184.189:38737)
0808 02:15:31.920423 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.920434 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.920450 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:15:31.920487 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.920491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.920544 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.920632 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.921597 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.922421 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.924135 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.924492 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.924593 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.927748 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.927754 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.927760 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.927787 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.929587 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.930115 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.930120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.930122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.930215 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.930572 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.930578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.930582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.930821 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.931052 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.931852 (+ 800us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.932392 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":141,"threads_started":1}
I20260808 02:15:31.932018 1316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } } }
I20260808 02:15:31.932904 1316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:15:31.933221 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.924215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:40807 (local address 127.0.0.1:51162)
0808 02:15:31.924443 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.924457 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:31.924525 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:31.924751 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:31.924757 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:31.924929 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:31.925018 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.925025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.925981 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.925986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.926630 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:31.926639 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.927169 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.927173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:31.927247 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.927716 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:31.927730 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:31.929354 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:31.931260 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.931263 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.931265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.931496 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.931785 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:31.931787 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:31.931789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:31.931942 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:31.932375 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:31.932377 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:31.932457 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 02:15:31.932547 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 02:15:31.932702 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260808 02:15:31.933785 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.924424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51162 (local address 127.18.184.188:40807)
0808 02:15:31.924529 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.924535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.924550 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:15:31.924586 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.924589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.924629 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.924705 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.925131 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.925831 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.926753 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.927062 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.927169 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.929498 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.929503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.929509 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.929535 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.931153 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.931596 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.931600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.931601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.931681 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.932040 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.932044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.932046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.932247 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.932338 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.933491 (+ 1153us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.933614 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:15:31.935112 1140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6: No bootstrap required, opened a new log
I20260808 02:15:31.936393 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:15:31.936721 1306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099: Bootstrap starting.
I20260808 02:15:31.939642 1140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } }
I20260808 02:15:31.939992 1140 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:31.939997 1306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:31.940114 1140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b6856a973d64fc0b5eddc0d0e07f2b6, State: Initialized, Role: FOLLOWER
W20260808 02:15:31.940671 1329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 02:15:31.940689 1140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } }
W20260808 02:15:31.940833 1329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:15:31.941013 1306 log.cc:824] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:31.942224 1140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:31.938753 1147 master_runner.cc:186] Error getting master registration for 127.18.184.190:41605: OK, instance_id { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" instance_seqno: 1786155331378327 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:15:31.943027 1147 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:15:31.943656 1306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099: No bootstrap required, opened a new log
W20260808 02:15:31.948127 1351 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:15:31.948304 1351 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:15:31.948553 1306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } }
I20260808 02:15:31.948925 1306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:31.949044 1306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19fe28805eb04cb7b82654eeb9d79099, State: Initialized, Role: FOLLOWER
I20260808 02:15:31.949733 1306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } }
I20260808 02:15:31.950340 1335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } } }
I20260808 02:15:31.951457 1306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:31.950481 1069 master_runner.cc:186] Error getting master registration for 127.18.184.188:40807: OK, instance_id { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" instance_seqno: 1786155331878987 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:15:31.954105 1069 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:15:31.957996 1353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } } }
I20260808 02:15:31.958858 1353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:15:31.959678 1365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:15:31.959833 1365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:15:31.950487 1335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:15:31.960768 1229 master_runner.cc:263] No leader master found from master 19fe28805eb04cb7b82654eeb9d79099
I20260808 02:15:31.961006 1229 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:15:31.968093 1315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.943558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46342 (local address 127.18.184.189:38737)
0808 02:15:31.943731 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.943739 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.948719 (+ 4980us) server_negotiation.cc:407] Connection header received
0808 02:15:31.948934 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.948938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.949031 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.949090 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.950640 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.951451 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.954087 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.954460 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.954540 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.961308 (+ 6768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.961312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.961315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.961345 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.963657 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.964485 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.964489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.964492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.964594 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.965900 (+ 1306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.965903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.965905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.966166 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.966302 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.967690 (+ 1388us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.967912 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:15:31.976102 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.944129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48944 (local address 127.18.184.190:41605)
0808 02:15:31.944294 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.944300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.952837 (+ 8537us) server_negotiation.cc:407] Connection header received
0808 02:15:31.953009 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.953012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.953060 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.953108 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.957686 (+ 4578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.958424 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.961771 (+ 3347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.962163 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.962250 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.969993 (+ 7743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.969997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.970000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.970034 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.972211 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.973981 (+ 1770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.973986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.973988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.974090 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.974460 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.974464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.974466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.974743 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.974867 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.975773 (+ 906us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.975921 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 02:15:31.977965 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.944412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51168 (local address 127.18.184.188:40807)
0808 02:15:31.944537 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:31.944542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:31.955300 (+ 10758us) server_negotiation.cc:407] Connection header received
0808 02:15:31.955522 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:31.955525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:31.955573 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:31.955626 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:31.959507 (+ 3881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.960182 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.966642 (+ 6460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:31.967010 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:31.967097 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:31.970468 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:31.970474 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:31.970478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:31.970511 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:31.973007 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.975252 (+ 2245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.975256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.975258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.975377 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:31.976875 (+ 1498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:31.976879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:31.976880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:31.977117 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:31.977289 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:31.977675 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 02:15:31.977816 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:15:32.076030 1353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:32.076310 1353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } }
I20260808 02:15:32.077528 1353 leader_election.cc:288] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b6856a973d64fc0b5eddc0d0e07f2b6 (127.18.184.190:41605), 27e9482f10c04abfa200356834980a64 (127.18.184.189:38737)
I20260808 02:15:32.078176 1112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19fe28805eb04cb7b82654eeb9d79099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" is_pre_election: true
I20260808 02:15:32.078523 1112 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19fe28805eb04cb7b82654eeb9d79099 in term 0.
I20260808 02:15:32.079306 1196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19fe28805eb04cb7b82654eeb9d79099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27e9482f10c04abfa200356834980a64" is_pre_election: true
I20260808 02:15:32.079615 1196 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19fe28805eb04cb7b82654eeb9d79099 in term 0.
I20260808 02:15:32.080336 1250 leader_election.cc:302] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19fe28805eb04cb7b82654eeb9d79099, 27e9482f10c04abfa200356834980a64; no voters:
I20260808 02:15:32.080896 1353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:15:32.080997 1353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:15:32.081041 1353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:32.082552 1353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } }
I20260808 02:15:32.083503 1196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19fe28805eb04cb7b82654eeb9d79099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27e9482f10c04abfa200356834980a64"
I20260808 02:15:32.083665 1196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:32.085240 1196 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19fe28805eb04cb7b82654eeb9d79099 in term 1.
I20260808 02:15:32.085701 1250 leader_election.cc:302] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19fe28805eb04cb7b82654eeb9d79099, 27e9482f10c04abfa200356834980a64; no voters:
I20260808 02:15:32.086664 1112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19fe28805eb04cb7b82654eeb9d79099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6"
I20260808 02:15:32.086818 1112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:32.088224 1112 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19fe28805eb04cb7b82654eeb9d79099 in term 1.
I20260808 02:15:32.088727 1353 leader_election.cc:288] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [CANDIDATE]: Term 1 election: Requested vote from peers 6b6856a973d64fc0b5eddc0d0e07f2b6 (127.18.184.190:41605), 27e9482f10c04abfa200356834980a64 (127.18.184.189:38737)
I20260808 02:15:32.088967 1353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:32.089123 1353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [term 1 LEADER]: Becoming Leader. State: Replica: 19fe28805eb04cb7b82654eeb9d79099, State: Running, Role: LEADER
I20260808 02:15:32.089553 1353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } }
I20260808 02:15:32.091723 1353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19fe28805eb04cb7b82654eeb9d79099. Latest consensus state: current_term: 1 leader_uuid: "19fe28805eb04cb7b82654eeb9d79099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } } }
I20260808 02:15:32.091933 1353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:32.097121 1369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:32.098276 1369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:32.104627 1196 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [term 1 FOLLOWER]: Refusing update from remote peer 19fe28805eb04cb7b82654eeb9d79099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:32.105693 1367 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:32.110222 1316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19fe28805eb04cb7b82654eeb9d79099. Latest consensus state: current_term: 1 leader_uuid: "19fe28805eb04cb7b82654eeb9d79099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } } }
I20260808 02:15:32.110364 1316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:15:32.114167 1112 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [term 1 FOLLOWER]: Refusing update from remote peer 19fe28805eb04cb7b82654eeb9d79099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:32.114857 1367 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:32.118788 1335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19fe28805eb04cb7b82654eeb9d79099. Latest consensus state: current_term: 1 leader_uuid: "19fe28805eb04cb7b82654eeb9d79099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } } }
I20260808 02:15:32.118919 1335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:15:32.124929 1253 tablet.cc:2369] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:15:32.124964 1091 tablet.cc:2369] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:15:32.124979 1172 tablet.cc:2369] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:15:32.126391 1367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "19fe28805eb04cb7b82654eeb9d79099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } } }
I20260808 02:15:32.126533 1367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:32.130625 1335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "19fe28805eb04cb7b82654eeb9d79099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } } }
I20260808 02:15:32.130801 1335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:15:32.131708 1370 mvcc.cc:205] Tried to move back new op lower bound from 7316092240291012608 to 7316092240243060736. Current Snapshot: MvccSnapshot[applied={T|T < 7316092240291012608}]
I20260808 02:15:32.132838 1353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "19fe28805eb04cb7b82654eeb9d79099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } } }
I20260808 02:15:32.132959 1353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:32.134369 1316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "19fe28805eb04cb7b82654eeb9d79099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6856a973d64fc0b5eddc0d0e07f2b6" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41605 } } peers { permanent_uuid: "27e9482f10c04abfa200356834980a64" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 38737 } } peers { permanent_uuid: "19fe28805eb04cb7b82654eeb9d79099" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 40807 } } }
I20260808 02:15:32.134496 1316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:15:32.135385 1373 mvcc.cc:205] Tried to move back new op lower bound from 7316092240291012608 to 7316092240243060736. Current Snapshot: MvccSnapshot[applied={T|T < 7316092240291012608}]
I20260808 02:15:32.135671 1371 mvcc.cc:205] Tried to move back new op lower bound from 7316092240291012608 to 7316092240243060736. Current Snapshot: MvccSnapshot[applied={T|T < 7316092240291012608}]
I20260808 02:15:32.139628 1369 catalog_manager.cc:1347] Generated new cluster ID: b92d5191ba4d4e6691fad2522f6ddd9f
I20260808 02:15:32.139716 1369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:15:32.156291 1369 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:15:32.157791 1369 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:32.168639 1369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19fe28805eb04cb7b82654eeb9d79099: Generated new TSK 0
I20260808 02:15:32.170140 1369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:15:32.942471 1329 catalog_manager.cc:1259] Loaded cluster ID: b92d5191ba4d4e6691fad2522f6ddd9f
I20260808 02:15:32.942582 1329 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64: loading cluster ID for follower catalog manager: success
I20260808 02:15:32.944384 1329 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64: acquiring CA information for follower catalog manager: success
I20260808 02:15:32.945675 1329 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 27e9482f10c04abfa200356834980a64: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:15:32.949565 1351 catalog_manager.cc:1259] Loaded cluster ID: b92d5191ba4d4e6691fad2522f6ddd9f
I20260808 02:15:32.949630 1351 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6: loading cluster ID for follower catalog manager: success
I20260808 02:15:32.951206 1351 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6: acquiring CA information for follower catalog manager: success
I20260808 02:15:32.952375 1351 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6b6856a973d64fc0b5eddc0d0e07f2b6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:15:33.245477 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 1069
I20260808 02:15:33.254978 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 1147
I20260808 02:15:33.263827 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 1229
2026-08-08T02:15:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3306 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:15:33 dist-test-slave-9gcw krb5kdc[1384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:15:33 dist-test-slave-9gcw krb5kdc[1384](info): set up 2 sockets
Aug 08 02:15:33 dist-test-slave-9gcw krb5kdc[1384](info): commencing operation
krb5kdc: starting...
W20260808 02:15:34.408872 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:15:34 dist-test-slave-9gcw krb5kdc[1384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:15:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:15:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:15:34.549607 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:42761
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:42761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:34.666123 1398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:34.666450 1398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:34.666496 1398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:34.670821 1398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:34.670898 1398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:34.670918 1398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:34.670936 1398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:34.670975 1398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:34.676734 1398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:42761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:42761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:34.677974 1398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:34.679075 1398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:34.688306 1404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:34.688366 1403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:34.688378 1406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:34.688732 1398 server_base.cc:1034] running on GCE node
I20260808 02:15:34.689652 1398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:34.690861 1398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:34.692067 1398 hybrid_clock.cc:648] HybridClock initialized: now 1786155334692035 us; error 54 us; skew 500 ppm
Aug 08 02:15:34 dist-test-slave-9gcw krb5kdc[1384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155334, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:34.695088 1398 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:15:34.696054 1398 webserver.cc:460] Webserver started at http://127.18.184.190:45683/ using document root <none> and password file <none>
I20260808 02:15:34.696635 1398 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:34.696734 1398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:34.696986 1398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:34.699256 1398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "8f35b7481133407ea678593457e7e61b"
format_stamp: "Formatted at 2026-08-08 02:15:34 on dist-test-slave-9gcw"
server_key: "8254939f7c9856d73c52e42f6a5df669"
server_key_iv: "37b4ff6a2d547e9ff57554c037f9741f"
server_key_version: "encryptionkey@0"
I20260808 02:15:34.699867 1398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "8f35b7481133407ea678593457e7e61b"
format_stamp: "Formatted at 2026-08-08 02:15:34 on dist-test-slave-9gcw"
server_key: "8254939f7c9856d73c52e42f6a5df669"
server_key_iv: "37b4ff6a2d547e9ff57554c037f9741f"
server_key_version: "encryptionkey@0"
I20260808 02:15:34.703948 1398 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:15:34.706635 1413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:34.708377 1398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:15:34.708554 1398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "8f35b7481133407ea678593457e7e61b"
format_stamp: "Formatted at 2026-08-08 02:15:34 on dist-test-slave-9gcw"
server_key: "8254939f7c9856d73c52e42f6a5df669"
server_key_iv: "37b4ff6a2d547e9ff57554c037f9741f"
server_key_version: "encryptionkey@0"
I20260808 02:15:34.708705 1398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:34.731078 1398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:34.734512 1398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:34.734735 1398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:34.742543 1398 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:42761
I20260808 02:15:34.742581 1470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:42761 every 8 connection(s)
I20260808 02:15:34.743562 1398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:34.746587 1471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:34.748281 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 1398
I20260808 02:15:34.748445 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:15:34.748792 19170 external_mini_cluster.cc:1428] Setting key a87eb9b556b27cfd1678ce054077dc43
I20260808 02:15:34.753994 1471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b: Bootstrap starting.
I20260808 02:15:34.756784 1471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:34.757779 1471 log.cc:824] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:15:34 dist-test-slave-9gcw krb5kdc[1384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:34.760135 1471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b: No bootstrap required, opened a new log
I20260808 02:15:34.765175 1471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f35b7481133407ea678593457e7e61b" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42761 } }
I20260808 02:15:34.765483 1471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:34.765532 1471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f35b7481133407ea678593457e7e61b, State: Initialized, Role: FOLLOWER
I20260808 02:15:34.766047 1474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:34.750337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43288 (local address 127.18.184.190:42761)
0808 02:15:34.750844 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:34.750863 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:34.750909 (+ 46us) server_negotiation.cc:407] Connection header received
0808 02:15:34.751758 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:34.751795 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:34.752231 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:34.752621 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:34.753505 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:34.754998 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:34.756191 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:34.756928 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:34.757120 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:34.760066 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:34.760072 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:34.760088 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:34.760120 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:34.762428 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:34.763043 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:34.763049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:34.763055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:34.763204 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:34.763532 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:34.763535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:34.763537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:34.764098 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:34.764310 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:34.764635 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 02:15:34.764964 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":153,"threads_started":1}
I20260808 02:15:34.766012 1471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f35b7481133407ea678593457e7e61b" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42761 } }
I20260808 02:15:34.766252 1471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:34.766337 1471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:34.766438 1471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:34.767624 1471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f35b7481133407ea678593457e7e61b" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42761 } }
I20260808 02:15:34.767999 1471 leader_election.cc:302] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f35b7481133407ea678593457e7e61b; no voters:
I20260808 02:15:34.768411 1471 leader_election.cc:288] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:15:34.768558 1476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:34.768816 1476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [term 1 LEADER]: Becoming Leader. State: Replica: 8f35b7481133407ea678593457e7e61b, State: Running, Role: LEADER
I20260808 02:15:34.769179 1476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f35b7481133407ea678593457e7e61b" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42761 } }
I20260808 02:15:34.769560 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:15:34.769850 1471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:34.771152 1478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f35b7481133407ea678593457e7e61b. Latest consensus state: current_term: 1 leader_uuid: "8f35b7481133407ea678593457e7e61b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f35b7481133407ea678593457e7e61b" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42761 } } }
I20260808 02:15:34.771292 1478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:34.771711 1477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f35b7481133407ea678593457e7e61b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f35b7481133407ea678593457e7e61b" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42761 } } }
I20260808 02:15:34.771878 1477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b [sys.catalog]: This master's current role is: LEADER
W20260808 02:15:34.774686 1493 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:15:34.774757 1493 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:15:34.774839 1487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:34.775642 1487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:34.781693 1487 catalog_manager.cc:1347] Generated new cluster ID: 7631544e9df2485887739e07f1f1c560
I20260808 02:15:34.781774 1487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:15:34.806725 1487 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:15:34.807701 1487 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:34.813436 1487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f35b7481133407ea678593457e7e61b: Generated new TSK 0
I20260808 02:15:34.814282 1487 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:15:34.826228 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 1398
2026-08-08T02:15:34Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1559 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:15:34 dist-test-slave-9gcw krb5kdc[1501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:15:34 dist-test-slave-9gcw krb5kdc[1501](info): set up 2 sockets
Aug 08 02:15:34 dist-test-slave-9gcw krb5kdc[1501](info): commencing operation
krb5kdc: starting...
W20260808 02:15:35.944483 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:15:36 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:15:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:15:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:15:36.084558 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34007
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:34007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:36.200235 1514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:36.200533 1514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:36.200575 1514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:36.204715 1514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:36.204859 1514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:36.204883 1514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:36.204902 1514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:36.204918 1514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:36.210630 1514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:34007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:36.211813 1514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:36.212917 1514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:36.222029 1523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:36.222029 1521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:36.222272 1514 server_base.cc:1034] running on GCE node
W20260808 02:15:36.222067 1520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:36.222949 1514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:36.224035 1514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:36.225255 1514 hybrid_clock.cc:648] HybridClock initialized: now 1786155336225211 us; error 46 us; skew 500 ppm
Aug 08 02:15:36 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155336, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:36.228138 1514 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:15:36.229108 1514 webserver.cc:460] Webserver started at http://127.18.184.190:41775/ using document root <none> and password file <none>
I20260808 02:15:36.229729 1514 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:36.229825 1514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:36.230077 1514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:36.232239 1514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "b1e78155df354d9f8b986cf60daf5d2e"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "6b17bdeb27d7b3c5d96ebf390a33810c"
server_key_iv: "ba2621a9dd1dc272636dff7148590c91"
server_key_version: "encryptionkey@0"
I20260808 02:15:36.232849 1514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "b1e78155df354d9f8b986cf60daf5d2e"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "6b17bdeb27d7b3c5d96ebf390a33810c"
server_key_iv: "ba2621a9dd1dc272636dff7148590c91"
server_key_version: "encryptionkey@0"
I20260808 02:15:36.236896 1514 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 02:15:36.239651 1530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:36.240964 1514 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:15:36.241119 1514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "b1e78155df354d9f8b986cf60daf5d2e"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "6b17bdeb27d7b3c5d96ebf390a33810c"
server_key_iv: "ba2621a9dd1dc272636dff7148590c91"
server_key_version: "encryptionkey@0"
I20260808 02:15:36.241304 1514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:36.261775 1514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:36.265271 1514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:36.265502 1514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:36.273226 1514 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:34007
I20260808 02:15:36.273541 1587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:34007 every 8 connection(s)
I20260808 02:15:36.274399 1514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:36.277385 1588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:36.282531 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 1514
I20260808 02:15:36.282691 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:15:36.283011 19170 external_mini_cluster.cc:1428] Setting key 413d97c10dfd99eff34495132019ab26
I20260808 02:15:36.283401 1588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Bootstrap starting.
I20260808 02:15:36.286396 1588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:36.287349 1588 log.cc:824] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:36.289465 1588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: No bootstrap required, opened a new log
Aug 08 02:15:36 dist-test-slave-9gcw krb5kdc[1501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:36.294487 1588 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } }
I20260808 02:15:36.294759 1588 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:36.294824 1588 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1e78155df354d9f8b986cf60daf5d2e, State: Initialized, Role: FOLLOWER
I20260808 02:15:36.295367 1588 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } }
I20260808 02:15:36.295533 1588 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:36.295610 1588 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:36.295756 1588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:36.296741 1588 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } }
I20260808 02:15:36.297102 1588 leader_election.cc:302] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1e78155df354d9f8b986cf60daf5d2e; no voters:
I20260808 02:15:36.297580 1588 leader_election.cc:288] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:15:36.297668 1594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:36.297875 1594 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 1 LEADER]: Becoming Leader. State: Replica: b1e78155df354d9f8b986cf60daf5d2e, State: Running, Role: LEADER
I20260808 02:15:36.298255 1594 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } }
I20260808 02:15:36.298492 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:36.284500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.18.184.190:34007)
0808 02:15:36.284989 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:36.285009 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:36.285055 (+ 46us) server_negotiation.cc:407] Connection header received
0808 02:15:36.285956 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:36.285981 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:36.286425 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:36.286831 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:36.287541 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:36.288551 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:36.289529 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:36.290061 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:36.290239 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:36.293056 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:36.293061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:36.293075 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:36.293104 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:36.295070 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:36.295558 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:36.295571 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:36.295576 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:36.295785 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:36.296092 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:36.296095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:36.296097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:36.296524 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:36.296728 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:36.297044 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 02:15:36.297411 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":141,"threads_started":1}
I20260808 02:15:36.298959 1588 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:36.299952 1595 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1e78155df354d9f8b986cf60daf5d2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } } }
I20260808 02:15:36.300102 1595 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:36.300931 1603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:36.299515 1594 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1e78155df354d9f8b986cf60daf5d2e. Latest consensus state: current_term: 1 leader_uuid: "b1e78155df354d9f8b986cf60daf5d2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } } }
I20260808 02:15:36.301501 1594 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:36.304179 1603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:36.310521 1603 catalog_manager.cc:1347] Generated new cluster ID: 8a56baa8969249a090122ad88dd96f4f
I20260808 02:15:36.310618 1603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:15:36.330849 1603 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:15:36.331979 1603 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:36.344357 1603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Generated new TSK 0
I20260808 02:15:36.345286 1603 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:15:36.388867 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34007
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:36.506151 1615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:36.506438 1615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:36.506484 1615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:36.510586 1615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:36.510663 1615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:36.510771 1615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:15:36.516403 1615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:36.517664 1615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:36.518720 1615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:36.529012 1622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:36.529085 1615 server_base.cc:1034] running on GCE node
W20260808 02:15:36.529032 1625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:36.529052 1623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:36.529803 1615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:36.530489 1615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:36.531687 1615 hybrid_clock.cc:648] HybridClock initialized: now 1786155336531679 us; error 55 us; skew 500 ppm
Aug 08 02:15:36 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155336, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:36.534576 1615 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:15:36.535734 1615 webserver.cc:460] Webserver started at http://127.18.184.129:45943/ using document root <none> and password file <none>
I20260808 02:15:36.536320 1615 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:36.536382 1615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:36.536553 1615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:36.538630 1615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "6831a9d158414ab192181b4aa3223d0c"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "bab146268cb40b7f4ca5f01d4ba0d89b"
server_key_iv: "a11c24e2062d17add38555ae31260a44"
server_key_version: "encryptionkey@0"
I20260808 02:15:36.539139 1615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "6831a9d158414ab192181b4aa3223d0c"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "bab146268cb40b7f4ca5f01d4ba0d89b"
server_key_iv: "a11c24e2062d17add38555ae31260a44"
server_key_version: "encryptionkey@0"
I20260808 02:15:36.542847 1615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 02:15:36.545243 1632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:36.546555 1615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 02:15:36.546701 1615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "6831a9d158414ab192181b4aa3223d0c"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "bab146268cb40b7f4ca5f01d4ba0d89b"
server_key_iv: "a11c24e2062d17add38555ae31260a44"
server_key_version: "encryptionkey@0"
I20260808 02:15:36.546810 1615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:36.592736 1615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:36.596144 1615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:36.596383 1615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:36.597110 1615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:36.598013 1615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:36.598089 1615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:36.598165 1615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:36.598208 1615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:36.608215 1615 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:40941
I20260808 02:15:36.608238 1753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:40941 every 8 connection(s)
I20260808 02:15:36.609261 1615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:15:36.617288 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 1615
I20260808 02:15:36.617439 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:15:36.617720 19170 external_mini_cluster.cc:1428] Setting key 909b6c0ca69e2155668fda37618af2b1
Aug 08 02:15:36 dist-test-slave-9gcw krb5kdc[1501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155336, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:36.624079 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:36.611378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:55209 (local address 127.18.184.190:34007)
0808 02:15:36.611533 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:36.611537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:36.612378 (+ 841us) server_negotiation.cc:407] Connection header received
0808 02:15:36.613883 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:36.613887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:36.613951 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:36.614061 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:36.615304 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:36.615885 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:36.617057 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:36.617386 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:36.617487 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:36.620347 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:36.620351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:36.620354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:36.620381 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:36.621914 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:36.622472 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:36.622475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:36.622477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:36.622564 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:36.622922 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:36.622927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:36.622929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:36.623122 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:36.623231 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:36.623776 (+ 545us) server_negotiation.cc:299] Negotiation successful
0808 02:15:36.623911 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:15:36.624959 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:36.611745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:55209)
0808 02:15:36.612212 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:36.612255 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:36.613637 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:36.614234 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:36.614241 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:36.614650 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:36.615142 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:36.615156 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:36.616030 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:36.616033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:36.616917 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:36.616926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:36.617544 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:36.617548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:36.617715 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:36.618403 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:36.618420 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:36.620134 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:36.622048 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:36.622054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:36.622072 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:36.622364 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:36.622662 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:36.622666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:36.622668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:36.622822 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:36.623244 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:36.623252 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:36.623526 (+ 274us) client_negotiation.cc:769] Sending connection context
0808 02:15:36.623728 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 02:15:36.624078 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":115,"threads_started":1}
I20260808 02:15:36.626353 1754 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34007
I20260808 02:15:36.626619 1754 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:36.627139 1754 heartbeater.cc:511] Master 127.18.184.190:34007 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:15:36.628731 1547 ts_manager.cc:194] Registered new tserver with Master: 6831a9d158414ab192181b4aa3223d0c (127.18.184.129:40941)
I20260808 02:15:36.629961 1547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:55209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:15:36.648694 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34007
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:36.766775 1759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:36.767123 1759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:36.767203 1759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:36.771395 1759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:36.771510 1759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:36.771615 1759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:15:36.777486 1759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:36.778678 1759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:36.779788 1759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:36.790154 1768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:36.790154 1764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:36.790325 1759 server_base.cc:1034] running on GCE node
W20260808 02:15:36.790500 1765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:36.790923 1759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:36.791589 1759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:36.792793 1759 hybrid_clock.cc:648] HybridClock initialized: now 1786155336792751 us; error 55 us; skew 500 ppm
Aug 08 02:15:36 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155336, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:36.795866 1759 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:15:36.796881 1759 webserver.cc:460] Webserver started at http://127.18.184.130:36331/ using document root <none> and password file <none>
I20260808 02:15:36.797685 1759 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:36.797802 1759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:36.798034 1759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:36.800329 1759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "2a414042277e4eb6a6a11fcccfb98e75"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "6daf6a5a96af2fc1cbb40d27e4de924d"
server_key_iv: "30b3c2d49144987aa26d0d04a1fd1812"
server_key_version: "encryptionkey@0"
I20260808 02:15:36.800980 1759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "2a414042277e4eb6a6a11fcccfb98e75"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "6daf6a5a96af2fc1cbb40d27e4de924d"
server_key_iv: "30b3c2d49144987aa26d0d04a1fd1812"
server_key_version: "encryptionkey@0"
I20260808 02:15:36.805137 1759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20260808 02:15:36.807998 1775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:36.809706 1759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:36.809855 1759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "2a414042277e4eb6a6a11fcccfb98e75"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "6daf6a5a96af2fc1cbb40d27e4de924d"
server_key_iv: "30b3c2d49144987aa26d0d04a1fd1812"
server_key_version: "encryptionkey@0"
I20260808 02:15:36.809981 1759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:36.826932 1759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:36.830489 1759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:36.830730 1759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:36.831403 1759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:36.832326 1759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:36.832386 1759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:36.832475 1759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:36.832509 1759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:36.842700 1759 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:38831
I20260808 02:15:36.842724 1899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:38831 every 8 connection(s)
I20260808 02:15:36.843683 1759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:15:36.847193 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 1759
I20260808 02:15:36.847347 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:15:36.847702 19170 external_mini_cluster.cc:1428] Setting key 47854070bc8505ebe19e270dcef4b867
Aug 08 02:15:36 dist-test-slave-9gcw krb5kdc[1501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155336, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:15:36.860769 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:36.845775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:45541 (local address 127.18.184.190:34007)
0808 02:15:36.845981 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:36.845985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:36.846846 (+ 861us) server_negotiation.cc:407] Connection header received
0808 02:15:36.848643 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:36.848647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:36.848703 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:36.848822 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:36.850572 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:36.851423 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:36.852711 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:36.852953 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:36.853074 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:36.856551 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:36.856557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:36.856563 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:36.856602 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:36.858460 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:36.859041 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:36.859045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:36.859047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:36.859145 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:36.859503 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:36.859506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:36.859508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:36.859755 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:36.859873 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:36.860521 (+ 648us) server_negotiation.cc:299] Negotiation successful
0808 02:15:36.860636 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260808 02:15:36.861743 1902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:36.846222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:45541)
0808 02:15:36.846666 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:36.846710 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:36.848429 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:36.848995 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:36.849002 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:36.849548 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:36.850326 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:36.850351 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:36.851640 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:36.851647 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:36.852579 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:36.852589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:36.853120 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:36.853124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:36.853399 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:36.854144 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:36.854171 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:36.856341 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:36.858611 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:36.858616 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:36.858631 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:36.858921 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:36.859252 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:36.859254 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:36.859256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:36.859403 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:36.859863 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:36.859868 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:36.860250 (+ 382us) client_negotiation.cc:769] Sending connection context
0808 02:15:36.860452 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 02:15:36.860734 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":102,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
I20260808 02:15:36.863231 1900 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34007
I20260808 02:15:36.863505 1900 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:36.864189 1900 heartbeater.cc:511] Master 127.18.184.190:34007 requested a full tablet report, sending...
I20260808 02:15:36.865574 1547 ts_manager.cc:194] Registered new tserver with Master: 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831)
I20260808 02:15:36.866496 1547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:45541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:15:36.880775 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34007
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:37.000383 1909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:37.000743 1909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:37.000830 1909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:37.005549 1909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:37.005635 1909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:37.005759 1909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:15:37.011890 1909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:37.013239 1909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:37.014678 1909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:37.025555 1914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:37.025749 1909 server_base.cc:1034] running on GCE node
W20260808 02:15:37.025576 1915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:37.025588 1917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:37.026265 1909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:37.026867 1909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:37.028054 1909 hybrid_clock.cc:648] HybridClock initialized: now 1786155337028026 us; error 38 us; skew 500 ppm
Aug 08 02:15:37 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155337, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:37.031009 1909 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:15:37.031941 1909 webserver.cc:460] Webserver started at http://127.18.184.131:39359/ using document root <none> and password file <none>
I20260808 02:15:37.032532 1909 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:37.032629 1909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:37.032888 1909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:37.035112 1909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "5e54dd0d65694a849e0f3b1f16bc43cc"
format_stamp: "Formatted at 2026-08-08 02:15:37 on dist-test-slave-9gcw"
server_key: "5dcabd45d00ace439bbe97c2b7cf944e"
server_key_iv: "fcece7348cb29f98ffddec70e2f67a2f"
server_key_version: "encryptionkey@0"
I20260808 02:15:37.035737 1909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "5e54dd0d65694a849e0f3b1f16bc43cc"
format_stamp: "Formatted at 2026-08-08 02:15:37 on dist-test-slave-9gcw"
server_key: "5dcabd45d00ace439bbe97c2b7cf944e"
server_key_iv: "fcece7348cb29f98ffddec70e2f67a2f"
server_key_version: "encryptionkey@0"
I20260808 02:15:37.039717 1909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:15:37.042286 1924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:37.043541 1909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 02:15:37.043679 1909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "5e54dd0d65694a849e0f3b1f16bc43cc"
format_stamp: "Formatted at 2026-08-08 02:15:37 on dist-test-slave-9gcw"
server_key: "5dcabd45d00ace439bbe97c2b7cf944e"
server_key_iv: "fcece7348cb29f98ffddec70e2f67a2f"
server_key_version: "encryptionkey@0"
I20260808 02:15:37.043799 1909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:37.060150 1909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:37.063455 1909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:37.063687 1909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:37.064330 1909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:37.065306 1909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:37.065376 1909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:37.065452 1909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:37.065500 1909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:37.075047 1909 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:42337
I20260808 02:15:37.075065 2043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:42337 every 8 connection(s)
I20260808 02:15:37.076092 1909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:15:37.078774 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 1909
I20260808 02:15:37.078908 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:15:37.079262 19170 external_mini_cluster.cc:1428] Setting key 77e0976ffa20e469b194bde89de5be64
Aug 08 02:15:37 dist-test-slave-9gcw krb5kdc[1501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155337, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:37.090482 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.078025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:52623 (local address 127.18.184.190:34007)
0808 02:15:37.078192 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:37.078197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:37.079216 (+ 1019us) server_negotiation.cc:407] Connection header received
0808 02:15:37.081016 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:37.081020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:37.081073 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:37.081213 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:37.082413 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.082949 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.084051 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.084257 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.084338 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.086879 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:37.086882 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:37.086884 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:37.086909 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:37.088332 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:37.088901 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:37.088903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:37.088904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:37.088977 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:37.089388 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:37.089390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:37.089391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:37.089583 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:37.089663 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:37.090241 (+ 578us) server_negotiation.cc:299] Negotiation successful
0808 02:15:37.090349 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:15:37.091406 2046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.078476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:52623)
0808 02:15:37.079024 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.079088 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:37.080793 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:37.081400 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:37.081410 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:37.081788 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:37.082279 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.082292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.083060 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.083062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.083941 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.083950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.084392 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.084394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.084535 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.085072 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:37.085089 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:37.086700 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:37.088455 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:37.088460 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:37.088474 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:37.088807 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:37.089060 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:37.089062 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:37.089064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:37.089302 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:37.089682 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:37.089687 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:37.090008 (+ 321us) client_negotiation.cc:769] Sending connection context
0808 02:15:37.090191 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 02:15:37.090488 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":122,"threads_started":1}
I20260808 02:15:37.092666 2044 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34007
I20260808 02:15:37.093030 2044 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:37.093595 2044 heartbeater.cc:511] Master 127.18.184.190:34007 requested a full tablet report, sending...
I20260808 02:15:37.094624 1546 ts_manager.cc:194] Registered new tserver with Master: 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337)
I20260808 02:15:37.095197 1546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:52623
I20260808 02:15:37.105561 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:15:37.115804 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.107754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40580 (local address 127.18.184.190:34007)
0808 02:15:37.107919 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:37.107924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:37.108174 (+ 250us) server_negotiation.cc:407] Connection header received
0808 02:15:37.108275 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:37.108277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:37.108322 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:37.108390 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:37.109184 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.109730 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.110671 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.110896 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.110974 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.112421 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:37.112423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:37.112426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:37.112445 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:37.113949 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:37.114414 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:37.114416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:37.114418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:37.114491 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:37.114956 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:37.114959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:37.114960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:37.115183 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:37.115294 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:37.115539 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 02:15:37.115638 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:15:37.118896 1546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:15:37.122439 1546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:15:37.138299 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.132071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:42337 (local address 127.0.0.1:54910)
0808 02:15:37.132538 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.132589 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:37.132726 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:37.133798 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:37.133804 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:37.133829 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:37.134028 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.134038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.135259 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.135265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.136644 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.136655 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.137675 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.137679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.137793 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.137844 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 02:15:37.137986 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 02:15:37.138121 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":28,"thread_start_us":92,"threads_started":1}
I20260808 02:15:37.138350 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.132492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:38831 (local address 127.0.0.1:44354)
0808 02:15:37.132794 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.132809 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:37.132906 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:37.134030 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:37.134033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:37.134081 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:37.134221 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.134228 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.135769 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.135773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.136942 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.136952 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.137974 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.137980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.138071 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.138094 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:15:37.138176 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 02:15:37.138230 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":35,"spinlock_wait_cycles":2944,"thread_start_us":93,"threads_started":1}
I20260808 02:15:37.138504 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.132794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44354 (local address 127.18.184.130:38831)
0808 02:15:37.133538 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:37.133552 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:37.133597 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:15:37.133676 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:37.133681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:37.133867 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:37.134397 (+ 530us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:37.134565 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.135651 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.137074 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.137859 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.137978 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.138099 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:37.138225 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 02:15:37.138325 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":114,"threads_started":1}
I20260808 02:15:37.138945 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.132207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54910 (local address 127.18.184.131:42337)
0808 02:15:37.133260 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:37.133275 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:37.133303 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:15:37.133389 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:37.133392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:37.133605 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:37.134192 (+ 587us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:37.134387 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.135119 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.136772 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.137546 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.137686 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.137831 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:37.138730 (+ 899us) server_negotiation.cc:299] Negotiation successful
0808 02:15:37.138835 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":895,"thread_start_us":168,"threads_started":1}
I20260808 02:15:37.139127 2063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.133093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40324 (local address 127.18.184.129:40941)
0808 02:15:37.133680 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:37.133686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:37.133704 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:15:37.133769 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:37.133773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:37.133929 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:37.134091 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:37.134364 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.135587 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.136969 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.137842 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.137978 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.138099 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:37.138889 (+ 790us) server_negotiation.cc:299] Negotiation successful
0808 02:15:37.138991 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":102,"threads_started":1}
I20260808 02:15:37.139412 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.132822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:40941 (local address 127.0.0.1:40324)
0808 02:15:37.133139 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.133175 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:37.133260 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:37.134062 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:37.134067 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:37.134101 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:37.134233 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.134240 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.135729 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.135733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.136853 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.136860 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.138189 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.138193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.138290 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.138315 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:15:37.138390 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 02:15:37.138461 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":26,"spinlock_wait_cycles":1920,"thread_start_us":96,"threads_started":1}
I20260808 02:15:37.141549 1831 tablet_service.cc:1467] Processing CreateTablet for tablet 948967362f774388ae35f9337e770bb3 (DEFAULT_TABLE table=test-table [id=784c20285eeb42089edea0a9433fff26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:37.141860 1975 tablet_service.cc:1467] Processing CreateTablet for tablet 948967362f774388ae35f9337e770bb3 (DEFAULT_TABLE table=test-table [id=784c20285eeb42089edea0a9433fff26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:37.142082 1684 tablet_service.cc:1467] Processing CreateTablet for tablet 948967362f774388ae35f9337e770bb3 (DEFAULT_TABLE table=test-table [id=784c20285eeb42089edea0a9433fff26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:37.142741 1831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 948967362f774388ae35f9337e770bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:37.142930 1684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 948967362f774388ae35f9337e770bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:37.142938 1975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 948967362f774388ae35f9337e770bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:37.150079 2064 tablet_bootstrap.cc:492] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75: Bootstrap starting.
I20260808 02:15:37.150904 2065 tablet_bootstrap.cc:492] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Bootstrap starting.
I20260808 02:15:37.151355 2066 tablet_bootstrap.cc:492] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc: Bootstrap starting.
I20260808 02:15:37.153671 2064 tablet_bootstrap.cc:654] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:37.153671 2066 tablet_bootstrap.cc:654] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:37.154918 2064 log.cc:824] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:37.154918 2066 log.cc:824] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:37.155110 2065 tablet_bootstrap.cc:654] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:37.156040 2065 log.cc:824] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:37.157219 2064 tablet_bootstrap.cc:492] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75: No bootstrap required, opened a new log
I20260808 02:15:37.157426 2064 ts_tablet_manager.cc:1397] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 02:15:37.157475 2066 tablet_bootstrap.cc:492] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc: No bootstrap required, opened a new log
I20260808 02:15:37.157637 2066 ts_tablet_manager.cc:1397] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:15:37.157902 2065 tablet_bootstrap.cc:492] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: No bootstrap required, opened a new log
I20260808 02:15:37.158090 2065 ts_tablet_manager.cc:1397] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 02:15:37.161523 2066 raft_consensus.cc:348] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } }
I20260808 02:15:37.161523 2064 raft_consensus.cc:348] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } }
I20260808 02:15:37.161542 2065 raft_consensus.cc:348] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } }
I20260808 02:15:37.161813 2066 raft_consensus.cc:374] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:37.161813 2064 raft_consensus.cc:374] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:37.161849 2065 raft_consensus.cc:374] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:37.161875 2064 raft_consensus.cc:729] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a414042277e4eb6a6a11fcccfb98e75, State: Initialized, Role: FOLLOWER
I20260808 02:15:37.161906 2066 raft_consensus.cc:729] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e54dd0d65694a849e0f3b1f16bc43cc, State: Initialized, Role: FOLLOWER
I20260808 02:15:37.161936 2065 raft_consensus.cc:729] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6831a9d158414ab192181b4aa3223d0c, State: Initialized, Role: FOLLOWER
I20260808 02:15:37.162426 2065 consensus_queue.cc:260] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } }
I20260808 02:15:37.162429 2066 consensus_queue.cc:260] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } }
I20260808 02:15:37.162429 2064 consensus_queue.cc:260] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } }
I20260808 02:15:37.162959 2071 raft_consensus.cc:482] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:37.163137 2071 raft_consensus.cc:504] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } }
I20260808 02:15:37.163445 2065 ts_tablet_manager.cc:1428] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 02:15:37.163447 2066 ts_tablet_manager.cc:1428] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:15:37.163447 2064 ts_tablet_manager.cc:1428] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:15:37.163820 2044 heartbeater.cc:503] Master 127.18.184.190:34007 was elected leader, sending a full tablet report...
I20260808 02:15:37.163820 1754 heartbeater.cc:503] Master 127.18.184.190:34007 was elected leader, sending a full tablet report...
I20260808 02:15:37.164425 2071 leader_election.cc:288] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337), 6831a9d158414ab192181b4aa3223d0c (127.18.184.129:40941)
I20260808 02:15:37.165481 1900 heartbeater.cc:503] Master 127.18.184.190:34007 was elected leader, sending a full tablet report...
I20260808 02:15:37.169303 1902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.164617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:42337 (local address 127.18.184.130:41771)
0808 02:15:37.165283 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.165300 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:37.165392 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:37.165744 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:37.165747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:37.165762 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:37.165871 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.165877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.166899 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.166903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.167978 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.167987 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.168850 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.168854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.168961 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.168980 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 02:15:37.169077 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 02:15:37.169151 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586}
I20260808 02:15:37.169307 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.165281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:41771 (local address 127.18.184.131:42337)
0808 02:15:37.165384 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:37.165388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:37.165402 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:15:37.165479 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:37.165482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:37.165530 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:37.165613 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:37.166010 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.166761 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.168128 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.168590 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.168727 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.168787 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:37.169094 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 02:15:37.169192 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:15:37.170137 1996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948967362f774388ae35f9337e770bb3" candidate_uuid: "2a414042277e4eb6a6a11fcccfb98e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" is_pre_election: true
I20260808 02:15:37.170459 1996 raft_consensus.cc:2454] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a414042277e4eb6a6a11fcccfb98e75 in term 0.
I20260808 02:15:37.171362 1778 leader_election.cc:302] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a414042277e4eb6a6a11fcccfb98e75, 5e54dd0d65694a849e0f3b1f16bc43cc; no voters:
I20260808 02:15:37.171543 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.164894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:40941 (local address 127.18.184.130:54407)
0808 02:15:37.166465 (+ 1571us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.166479 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:37.166616 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:37.167002 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:37.167005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:37.167021 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:37.167996 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.168005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.169120 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.169126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.170290 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.170303 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.171095 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.171098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.171195 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.171231 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:15:37.171314 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 02:15:37.171392 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1512,"mutex_wait_us":90,"thread_start_us":110,"threads_started":1}
I20260808 02:15:37.171548 2063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.165200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:54407 (local address 127.18.184.129:40941)
0808 02:15:37.165333 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:37.165337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:37.166551 (+ 1214us) server_negotiation.cc:407] Connection header received
0808 02:15:37.166739 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:37.166743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:37.166791 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:37.166847 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:37.168172 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.168871 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.170408 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.170965 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.171063 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.171099 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:37.171356 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 02:15:37.171420 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:15:37.171764 2070 raft_consensus.cc:2781] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:15:37.171873 2070 raft_consensus.cc:482] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:15:37.171924 2070 raft_consensus.cc:3037] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:37.172184 1705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948967362f774388ae35f9337e770bb3" candidate_uuid: "2a414042277e4eb6a6a11fcccfb98e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6831a9d158414ab192181b4aa3223d0c" is_pre_election: true
I20260808 02:15:37.172457 1705 raft_consensus.cc:2454] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a414042277e4eb6a6a11fcccfb98e75 in term 0.
I20260808 02:15:37.173072 2070 raft_consensus.cc:504] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } }
I20260808 02:15:37.173432 2070 leader_election.cc:288] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [CANDIDATE]: Term 1 election: Requested vote from peers 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337), 6831a9d158414ab192181b4aa3223d0c (127.18.184.129:40941)
I20260808 02:15:37.173760 1996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948967362f774388ae35f9337e770bb3" candidate_uuid: "2a414042277e4eb6a6a11fcccfb98e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc"
I20260808 02:15:37.173847 1705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948967362f774388ae35f9337e770bb3" candidate_uuid: "2a414042277e4eb6a6a11fcccfb98e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6831a9d158414ab192181b4aa3223d0c"
I20260808 02:15:37.173950 1705 raft_consensus.cc:3037] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:37.173943 1996 raft_consensus.cc:3037] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:37.175230 1996 raft_consensus.cc:2454] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a414042277e4eb6a6a11fcccfb98e75 in term 1.
I20260808 02:15:37.175230 1705 raft_consensus.cc:2454] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a414042277e4eb6a6a11fcccfb98e75 in term 1.
I20260808 02:15:37.175520 1778 leader_election.cc:302] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a414042277e4eb6a6a11fcccfb98e75, 6831a9d158414ab192181b4aa3223d0c; no voters:
I20260808 02:15:37.175714 2070 raft_consensus.cc:2781] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:37.175896 2070 raft_consensus.cc:686] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 1 LEADER]: Becoming Leader. State: Replica: 2a414042277e4eb6a6a11fcccfb98e75, State: Running, Role: LEADER
I20260808 02:15:37.176213 2070 consensus_queue.cc:237] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } }
I20260808 02:15:37.179677 1545 catalog_manager.cc:5495] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130). New cstate: current_term: 1 leader_uuid: "2a414042277e4eb6a6a11fcccfb98e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } health_report { overall_health: HEALTHY } } }
I20260808 02:15:37.203051 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.199198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44360 (local address 127.18.184.130:38831)
0808 02:15:37.199357 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:37.199363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:37.199378 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:15:37.199556 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:37.199559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:37.199618 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:37.199734 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:15:37.199986 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.200606 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.201597 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.201820 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.201932 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.202244 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:15:37.202659 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:15:37.202796 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:37.202842 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 02:15:37.202900 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:15:37.209106 1705 raft_consensus.cc:1261] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Refusing update from remote peer 2a414042277e4eb6a6a11fcccfb98e75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:37.209110 1996 raft_consensus.cc:1261] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 1 FOLLOWER]: Refusing update from remote peer 2a414042277e4eb6a6a11fcccfb98e75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:37.209991 2070 consensus_queue.cc:1035] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:37.210199 2071 consensus_queue.cc:1035] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:37.218019 2079 mvcc.cc:205] Tried to move back new op lower bound from 7316092261200941056 to 7316092261077004288. Current Snapshot: MvccSnapshot[applied={T|T < 7316092261200941056}]
I20260808 02:15:37.218158 2078 mvcc.cc:205] Tried to move back new op lower bound from 7316092261200941056 to 7316092261077004288. Current Snapshot: MvccSnapshot[applied={T|T < 7316092261200941056}]
I20260808 02:15:37.226570 1545 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:37.226720 1545 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:37.229269 1545 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Sending DeleteTablet for 3 replicas of tablet 948967362f774388ae35f9337e770bb3
I20260808 02:15:37.230149 1684 tablet_service.cc:1514] Processing DeleteTablet for tablet 948967362f774388ae35f9337e770bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:37 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:40324
I20260808 02:15:37.230157 1831 tablet_service.cc:1514] Processing DeleteTablet for tablet 948967362f774388ae35f9337e770bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:37 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:44354
I20260808 02:15:37.230182 1975 tablet_service.cc:1514] Processing DeleteTablet for tablet 948967362f774388ae35f9337e770bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:37 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:54910
I20260808 02:15:37.230672 2088 tablet_replica.cc:330] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75: stopping tablet replica
I20260808 02:15:37.230825 2087 tablet_replica.cc:330] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: stopping tablet replica
I20260808 02:15:37.230865 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 1615
I20260808 02:15:37.231070 2088 raft_consensus.cc:2229] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:15:37.237613 2089 tablet_replica.cc:330] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc: stopping tablet replica
I20260808 02:15:37.238366 2089 raft_consensus.cc:2229] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:37.238914 2089 raft_consensus.cc:2258] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:37.239153 2088 raft_consensus.cc:2258] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:37.241132 2088 ts_tablet_manager.cc:1905] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:37.241276 2089 ts_tablet_manager.cc:1905] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:37.244068 2088 ts_tablet_manager.cc:1918] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:37.244258 2088 log.cc:1196] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/948967362f774388ae35f9337e770bb3
I20260808 02:15:37.244712 2088 ts_tablet_manager.cc:1939] T 948967362f774388ae35f9337e770bb3 P 2a414042277e4eb6a6a11fcccfb98e75: Deleting consensus metadata
I20260808 02:15:37.244973 2089 ts_tablet_manager.cc:1918] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:37.245169 2089 log.cc:1196] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/948967362f774388ae35f9337e770bb3
I20260808 02:15:37.245596 2089 ts_tablet_manager.cc:1939] T 948967362f774388ae35f9337e770bb3 P 5e54dd0d65694a849e0f3b1f16bc43cc: Deleting consensus metadata
I20260808 02:15:37.246201 1534 catalog_manager.cc:4841] TS 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831): tablet 948967362f774388ae35f9337e770bb3 (table test-table [id=784c20285eeb42089edea0a9433fff26]) successfully deleted
I20260808 02:15:37.247133 1533 catalog_manager.cc:4841] TS 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337): tablet 948967362f774388ae35f9337e770bb3 (table test-table [id=784c20285eeb42089edea0a9433fff26]) successfully deleted
W20260808 02:15:37.250659 1533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:40941 (error 108)
I20260808 02:15:37.250900 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 1759
I20260808 02:15:37.254114 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.253776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:40941 (local address 127.0.0.1:40340)
0808 02:15:37.253954 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.254033 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 02:15:37.254325 1533 catalog_manager.cc:4584] TS 6831a9d158414ab192181b4aa3223d0c (127.18.184.129:40941): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 948967362f774388ae35f9337e770bb3: Network error: Client connection negotiation failed: client connection to 127.18.184.129:40941: connect: Connection refused (error 111)
I20260808 02:15:37.260198 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 1909
I20260808 02:15:37.267726 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 1514
I20260808 02:15:37.277354 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34007
--webserver_interface=127.18.184.190
--webserver_port=41775
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:34007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:37.394227 2091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:37.394583 2091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:37.394667 2091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:37.399092 2091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:37.399220 2091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:37.399257 2091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:37.399296 2091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:37.399338 2091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:37.404906 2091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:34007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=41775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:37.406181 2091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:37.407289 2091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:37.428113 2100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:37.428139 2098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:37.428438 2091 server_base.cc:1034] running on GCE node
W20260808 02:15:37.428113 2097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:37.429252 2091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:37.430651 2091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:37.431891 2091 hybrid_clock.cc:648] HybridClock initialized: now 1786155337431852 us; error 56 us; skew 500 ppm
Aug 08 02:15:37 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155337, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:37.435079 2091 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:15:37.436076 2091 webserver.cc:460] Webserver started at http://127.18.184.190:41775/ using document root <none> and password file <none>
I20260808 02:15:37.436671 2091 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:37.436748 2091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:37.440948 2091 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:37.443225 2107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:37.444633 2091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 02:15:37.444784 2091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "b1e78155df354d9f8b986cf60daf5d2e"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "6b17bdeb27d7b3c5d96ebf390a33810c"
server_key_iv: "ba2621a9dd1dc272636dff7148590c91"
server_key_version: "encryptionkey@0"
I20260808 02:15:37.445538 2091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:37.459739 2091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:37.463361 2091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:37.463555 2091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:37.471895 2091 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:34007
I20260808 02:15:37.472240 2163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:34007 every 8 connection(s)
I20260808 02:15:37.473603 2091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:37.475023 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 2091
I20260808 02:15:37.475544 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:40941
--local_ip_for_outbound_sockets=127.18.184.129
--tserver_master_addrs=127.18.184.190:34007
--webserver_port=45943
--webserver_interface=127.18.184.129
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:37.478618 2164 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:15:37.481237 2164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Bootstrap starting.
I20260808 02:15:37.506358 2164 log.cc:824] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:37.517792 2164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:15:37.518273 2164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Bootstrap complete.
I20260808 02:15:37.524369 2164 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } }
I20260808 02:15:37.524946 2164 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1e78155df354d9f8b986cf60daf5d2e, State: Initialized, Role: FOLLOWER
I20260808 02:15:37.525569 2164 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } }
I20260808 02:15:37.525727 2164 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:37.525803 2164 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:37.525909 2164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:15:37.528628 2164 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } }
I20260808 02:15:37.529022 2164 leader_election.cc:302] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1e78155df354d9f8b986cf60daf5d2e; no voters:
I20260808 02:15:37.529464 2164 leader_election.cc:288] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:15:37.529630 2169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:15:37.529927 2169 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 2 LEADER]: Becoming Leader. State: Replica: b1e78155df354d9f8b986cf60daf5d2e, State: Running, Role: LEADER
I20260808 02:15:37.530313 2169 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } }
I20260808 02:15:37.530822 2164 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:37.531963 2170 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b1e78155df354d9f8b986cf60daf5d2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } } }
I20260808 02:15:37.532104 2170 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:37.531948 2171 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1e78155df354d9f8b986cf60daf5d2e. Latest consensus state: current_term: 2 leader_uuid: "b1e78155df354d9f8b986cf60daf5d2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } } }
I20260808 02:15:37.532163 2171 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:37.532657 2179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:37.538578 2179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:37.539211 2179 catalog_manager.cc:1259] Loaded cluster ID: 8a56baa8969249a090122ad88dd96f4f
I20260808 02:15:37.539276 2179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:15:37.540938 2179 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 02:15:37.545462 2179 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 02:15:37.551522 2179 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:37.552425 2179 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Loaded TSK: 0
I20260808 02:15:37.553740 2179 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:15:37.616220 2166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:37.616508 2166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:37.616554 2166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:37.620743 2166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:37.620824 2166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:37.620887 2166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:15:37.626891 2166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:40941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=45943
--tserver_master_addrs=127.18.184.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:37.628064 2166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:37.629225 2166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:37.641172 2199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:37.641330 2196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:37.641173 2195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:37.641520 2166 server_base.cc:1034] running on GCE node
I20260808 02:15:37.642149 2166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:37.642863 2166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:37.644068 2166 hybrid_clock.cc:648] HybridClock initialized: now 1786155337644036 us; error 51 us; skew 500 ppm
Aug 08 02:15:37 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155337, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:37.647218 2166 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:15:37.648250 2166 webserver.cc:460] Webserver started at http://127.18.184.129:45943/ using document root <none> and password file <none>
I20260808 02:15:37.648967 2166 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:37.649092 2166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:37.653487 2166 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:37.656016 2207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:37.657462 2166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:37.657603 2166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "6831a9d158414ab192181b4aa3223d0c"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "bab146268cb40b7f4ca5f01d4ba0d89b"
server_key_iv: "a11c24e2062d17add38555ae31260a44"
server_key_version: "encryptionkey@0"
I20260808 02:15:37.658201 2166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:37.682605 2166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:37.685876 2166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:37.686044 2166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:37.686592 2166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:37.687640 2215 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:15:37.692162 2166 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:15:37.692271 2166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260808 02:15:37.692363 2166 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:15:37.694216 2166 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:15:37.694293 2166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.001s
I20260808 02:15:37.694608 2215 tablet_bootstrap.cc:492] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Bootstrap starting.
I20260808 02:15:37.704635 2166 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:40941
I20260808 02:15:37.704704 2328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:40941 every 8 connection(s)
I20260808 02:15:37.705719 2166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:15:37.715629 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 2166
I20260808 02:15:37.715791 2215 log.cc:824] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:37.716202 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:38831
--local_ip_for_outbound_sockets=127.18.184.130
--tserver_master_addrs=127.18.184.190:34007
--webserver_port=36331
--webserver_interface=127.18.184.130
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:15:37 dist-test-slave-9gcw krb5kdc[1501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155337, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:37.724366 2215 tablet_bootstrap.cc:492] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:15:37.724844 2215 tablet_bootstrap.cc:492] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Bootstrap complete.
I20260808 02:15:37.726445 2215 ts_tablet_manager.cc:1397] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.012s
I20260808 02:15:37.727207 2332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.707945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:44895)
0808 02:15:37.708973 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.709015 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:37.709885 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:37.711299 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:37.711305 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:37.711657 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:37.712137 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.712151 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.714166 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.714168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.715099 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.715110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.716163 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.716168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.716426 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.718368 (+ 1942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:37.718389 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:37.720377 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:37.723397 (+ 3020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:37.723403 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:37.723416 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:37.723782 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:37.724255 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:37.724258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:37.724260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:37.724437 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:37.725326 (+ 889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:37.725335 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:37.725641 (+ 306us) client_negotiation.cc:769] Sending connection context
0808 02:15:37.725852 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 02:15:37.726220 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":857,"thread_start_us":155,"threads_started":1}
I20260808 02:15:37.728058 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.708034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:44895 (local address 127.18.184.190:34007)
0808 02:15:37.708457 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:37.708485 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:37.709173 (+ 688us) server_negotiation.cc:407] Connection header received
0808 02:15:37.710387 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:37.710407 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:37.710865 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:37.711273 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:37.712333 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.714007 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.715241 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.716000 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.716171 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.720623 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:37.720632 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:37.720652 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:37.720694 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:37.723227 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:37.723933 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:37.723943 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:37.723951 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:37.724134 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:37.724565 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:37.724571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:37.724575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:37.725097 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:37.725267 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:37.726027 (+ 760us) server_negotiation.cc:299] Negotiation successful
0808 02:15:37.726420 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":143,"threads_started":1}
I20260808 02:15:37.729497 2215 raft_consensus.cc:348] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } }
I20260808 02:15:37.729930 2215 raft_consensus.cc:729] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6831a9d158414ab192181b4aa3223d0c, State: Initialized, Role: FOLLOWER
I20260808 02:15:37.730465 2329 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34007
I20260808 02:15:37.730466 2215 consensus_queue.cc:260] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } }
I20260808 02:15:37.730736 2329 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:37.731383 2329 heartbeater.cc:511] Master 127.18.184.190:34007 requested a full tablet report, sending...
I20260808 02:15:37.731791 2215 ts_tablet_manager.cc:1428] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 02:15:37.734014 2126 ts_manager.cc:194] Registered new tserver with Master: 6831a9d158414ab192181b4aa3223d0c (127.18.184.129:40941)
I20260808 02:15:37.737344 2126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:44895
I20260808 02:15:37.739869 2329 heartbeater.cc:503] Master 127.18.184.190:34007 was elected leader, sending a full tablet report...
Aug 08 02:15:37 dist-test-slave-9gcw krb5kdc[1501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155337, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.129@KRBTEST.COM
I20260808 02:15:37.748307 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.736877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40344 (local address 127.18.184.129:40941)
0808 02:15:37.737281 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:37.737293 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:37.737715 (+ 422us) server_negotiation.cc:407] Connection header received
0808 02:15:37.737907 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:37.737912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:37.738255 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:37.738409 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:37.739182 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.740374 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.741581 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.742017 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.742102 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.744710 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:37.744719 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:37.744729 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:37.744752 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:37.746388 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:37.746922 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:37.746926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:37.746928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:37.747007 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:37.747291 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:37.747293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:37.747294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:37.747549 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:37.747649 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:37.748037 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 02:15:37.748178 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":102,"threads_started":1}
I20260808 02:15:37.748373 2336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.736789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:40941 (local address 127.0.0.1:40344)
0808 02:15:37.737547 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.737592 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:37.737753 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:37.738466 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:37.738472 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:37.738915 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:37.739046 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.739055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.740489 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.740492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.741446 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.741456 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.742139 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.742143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.742256 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.742866 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:37.742882 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:37.744557 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:37.746501 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:37.746510 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:37.746516 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:37.746809 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:37.747095 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:37.747098 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:37.747100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:37.747194 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:37.747658 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:37.747664 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:37.747806 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 02:15:37.747964 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 02:15:37.748167 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":105,"threads_started":1}
I20260808 02:15:37.749483 2259 tablet_service.cc:1514] Processing DeleteTablet for tablet 948967362f774388ae35f9337e770bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:37 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:40344
I20260808 02:15:37.750516 2338 tablet_replica.cc:330] stopping tablet replica
I20260808 02:15:37.750845 2338 raft_consensus.cc:2229] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:37.751124 2338 raft_consensus.cc:2258] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:37.752239 2338 ts_tablet_manager.cc:1905] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:37.757203 2338 ts_tablet_manager.cc:1918] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:37.757345 2338 log.cc:1196] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal/wals/948967362f774388ae35f9337e770bb3
I20260808 02:15:37.757650 2338 ts_tablet_manager.cc:1939] T 948967362f774388ae35f9337e770bb3 P 6831a9d158414ab192181b4aa3223d0c: Deleting consensus metadata
I20260808 02:15:37.759770 2110 catalog_manager.cc:4841] TS 6831a9d158414ab192181b4aa3223d0c (127.18.184.129:40941): tablet 948967362f774388ae35f9337e770bb3 (table test-table [id=784c20285eeb42089edea0a9433fff26]) successfully deleted
W20260808 02:15:37.841741 2334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:37.842038 2334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:37.842082 2334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:37.846103 2334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:37.846165 2334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:37.846225 2334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:15:37.851969 2334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:38831
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=36331
--tserver_master_addrs=127.18.184.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:37.853065 2334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:37.854144 2334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:15:37.864567 2334 server_base.cc:1034] running on GCE node
W20260808 02:15:37.864533 2346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:37.864524 2345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:37.864524 2348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:37.865386 2334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:37.866111 2334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:37.867336 2334 hybrid_clock.cc:648] HybridClock initialized: now 1786155337867320 us; error 33 us; skew 500 ppm
Aug 08 02:15:37 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155337, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:37.870362 2334 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:15:37.871340 2334 webserver.cc:460] Webserver started at http://127.18.184.130:36331/ using document root <none> and password file <none>
I20260808 02:15:37.871941 2334 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:37.872062 2334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:37.876204 2334 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:15:37.878539 2356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:37.879911 2334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:15:37.880066 2334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "2a414042277e4eb6a6a11fcccfb98e75"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "6daf6a5a96af2fc1cbb40d27e4de924d"
server_key_iv: "30b3c2d49144987aa26d0d04a1fd1812"
server_key_version: "encryptionkey@0"
I20260808 02:15:37.880637 2334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:37.894969 2334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:37.898154 2334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:37.898377 2334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:37.899034 2334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:37.899927 2334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:37.899999 2334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:37.900076 2334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:37.900120 2334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:37.910375 2334 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:38831
I20260808 02:15:37.910421 2477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:38831 every 8 connection(s)
I20260808 02:15:37.911537 2334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:15:37.914880 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 2334
I20260808 02:15:37.915477 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:42337
--local_ip_for_outbound_sockets=127.18.184.131
--tserver_master_addrs=127.18.184.190:34007
--webserver_port=39359
--webserver_interface=127.18.184.131
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:15:37 dist-test-slave-9gcw krb5kdc[1501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155337, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:37.928117 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.913649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:36123 (local address 127.18.184.190:34007)
0808 02:15:37.913840 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:37.913845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:37.914801 (+ 956us) server_negotiation.cc:407] Connection header received
0808 02:15:37.916890 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:37.916893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:37.916950 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:37.917030 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:37.918566 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.919399 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.920635 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.921103 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.921272 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.923891 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:37.923897 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:37.923903 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:37.923941 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:37.925766 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:37.926293 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:37.926297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:37.926301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:37.926404 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:37.926745 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:37.926751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:37.926754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:37.927045 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:37.927158 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:37.927677 (+ 519us) server_negotiation.cc:299] Negotiation successful
0808 02:15:37.927857 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:15:37.928821 2480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.914091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:36123)
0808 02:15:37.914592 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.914667 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:37.916622 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:37.917304 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:37.917316 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:37.917877 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:37.918388 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.918402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.919553 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.919556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.920461 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:37.920470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:37.921325 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:37.921328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:37.921487 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:37.922146 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:37.922164 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:37.923690 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:37.925900 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:37.925906 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:37.925919 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:37.926184 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:37.926507 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:37.926509 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:37.926511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:37.926647 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:37.927163 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:37.927167 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:37.927425 (+ 258us) client_negotiation.cc:769] Sending connection context
0808 02:15:37.927586 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 02:15:37.927862 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":112,"threads_started":1}
I20260808 02:15:37.930243 2478 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34007
I20260808 02:15:37.930516 2478 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:37.931118 2478 heartbeater.cc:511] Master 127.18.184.190:34007 requested a full tablet report, sending...
I20260808 02:15:37.932437 2126 ts_manager.cc:194] Registered new tserver with Master: 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831)
I20260808 02:15:37.933043 2126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:36123
W20260808 02:15:38.040380 2481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:38.040689 2481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:38.040779 2481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:38.045039 2481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:38.045127 2481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:38.045307 2481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:15:38.051162 2481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:42337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=39359
--tserver_master_addrs=127.18.184.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:38.052376 2481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:38.053611 2481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:38.064275 2488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:38.064344 2487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:38.064344 2490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:38.064591 2481 server_base.cc:1034] running on GCE node
I20260808 02:15:38.065014 2481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:38.065706 2481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:38.066919 2481 hybrid_clock.cc:648] HybridClock initialized: now 1786155338066888 us; error 47 us; skew 500 ppm
Aug 08 02:15:38 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155338, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:38.069831 2481 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:15:38.070672 2481 webserver.cc:460] Webserver started at http://127.18.184.131:39359/ using document root <none> and password file <none>
I20260808 02:15:38.071203 2481 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:38.071275 2481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:38.075186 2481 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:15:38.077414 2497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:38.078729 2481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:15:38.078879 2481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "5e54dd0d65694a849e0f3b1f16bc43cc"
format_stamp: "Formatted at 2026-08-08 02:15:37 on dist-test-slave-9gcw"
server_key: "5dcabd45d00ace439bbe97c2b7cf944e"
server_key_iv: "fcece7348cb29f98ffddec70e2f67a2f"
server_key_version: "encryptionkey@0"
I20260808 02:15:38.079442 2481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:38.105348 2481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:38.108902 2481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:38.109117 2481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:38.109876 2481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:38.110750 2481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:38.110806 2481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:38.110872 2481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:38.110904 2481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:38.120302 2481 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:42337
I20260808 02:15:38.120337 2624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:42337 every 8 connection(s)
I20260808 02:15:38.121390 2481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:15:38.124255 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 2481
Aug 08 02:15:38 dist-test-slave-9gcw krb5kdc[1501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155338, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:38.134363 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.124991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40596 (local address 127.18.184.190:34007)
0808 02:15:38.125294 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:38.125297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:38.125492 (+ 195us) server_negotiation.cc:407] Connection header received
0808 02:15:38.125669 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:38.125671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:38.125716 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:38.125797 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:38.126774 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.127305 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.128416 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.128739 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.128812 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.130461 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:38.130465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:38.130468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:38.130495 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:38.132462 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:38.132989 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:38.132993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:38.132995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:38.133085 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:38.133432 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:38.133435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:38.133436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:38.133625 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:38.133897 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:38.134020 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 02:15:38.134165 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":85,"threads_started":1}
I20260808 02:15:38.137230 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.123306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:53763 (local address 127.18.184.190:34007)
0808 02:15:38.123459 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:38.123463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:38.124443 (+ 980us) server_negotiation.cc:407] Connection header received
0808 02:15:38.126462 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:38.126464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:38.126495 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:38.126544 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:38.127965 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.128547 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.130273 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.130553 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.130660 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.133322 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:38.133326 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:38.133329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:38.133354 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:38.135007 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:38.135612 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:38.135615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:38.135617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:38.135703 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:38.136044 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:38.136048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:38.136050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:38.136268 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:38.136385 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:38.136935 (+ 550us) server_negotiation.cc:299] Negotiation successful
0808 02:15:38.137053 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:15:38.138134 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.123704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:53763)
0808 02:15:38.124244 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:38.124318 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:38.126221 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:38.126758 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:38.126779 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:38.127171 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:38.127811 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:38.127831 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.128669 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.128672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:38.130138 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:38.130150 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.130708 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.130711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:38.130930 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.131488 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:38.131506 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:38.133125 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:38.135130 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:38.135135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:38.135148 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:38.135532 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:38.135791 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:38.135793 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:38.135794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:38.135943 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:38.136386 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:38.136394 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:38.136674 (+ 280us) client_negotiation.cc:769] Sending connection context
0808 02:15:38.136878 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 02:15:38.137251 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":136,"threads_started":1}
I20260808 02:15:38.139247 2625 heartbeater.cc:347] Connected to a master server at 127.18.184.190:34007
I20260808 02:15:38.139510 2625 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:38.140031 2625 heartbeater.cc:511] Master 127.18.184.190:34007 requested a full tablet report, sending...
I20260808 02:15:38.141018 2126 ts_manager.cc:194] Registered new tserver with Master: 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337)
I20260808 02:15:38.141621 2126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:53763
I20260808 02:15:38.148392 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:15:38.153029 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.149084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40608 (local address 127.18.184.190:34007)
0808 02:15:38.149248 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:38.149252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:38.149479 (+ 227us) server_negotiation.cc:407] Connection header received
0808 02:15:38.149634 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:38.149637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:38.149741 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:38.149838 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:15:38.150058 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.150775 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.151727 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.151945 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.152022 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.152289 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:15:38.152682 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:15:38.152773 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:38.152869 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 02:15:38.152916 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 02:15:38.153839 2126 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:15:38.157462 2126 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:15:38.165624 2259 tablet_service.cc:1467] Processing CreateTablet for tablet 45572eea5e5f4002821e5f0da00e6f56 (DEFAULT_TABLE table=test-table [id=2b827c1fbc9643848a2efee0d302526d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:38.166380 2259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45572eea5e5f4002821e5f0da00e6f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:38.169454 2215 tablet_bootstrap.cc:492] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c: Bootstrap starting.
I20260808 02:15:38.170491 2215 tablet_bootstrap.cc:654] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:38.170897 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.164751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:38831 (local address 127.0.0.1:44366)
0808 02:15:38.165612 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:38.165625 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:38.165751 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:38.166414 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:38.166420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:38.166442 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:38.166869 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:38.166880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.168331 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.168337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:38.169242 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:38.169261 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.170426 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.170431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:38.170522 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.170576 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 02:15:38.170656 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 02:15:38.170738 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":93,"thread_start_us":138,"threads_started":1}
I20260808 02:15:38.171043 2635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.165032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44366 (local address 127.18.184.130:38831)
0808 02:15:38.165450 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:38.165467 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:38.165733 (+ 266us) server_negotiation.cc:407] Connection header received
0808 02:15:38.165880 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:38.165886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:38.166056 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:38.166241 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:38.167020 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.168188 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.169436 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.170274 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.170448 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.170631 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:38.170703 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 02:15:38.170816 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":180,"threads_started":1}
I20260808 02:15:38.171052 2336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.164571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:42337 (local address 127.0.0.1:54918)
0808 02:15:38.164782 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:38.164800 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:38.164906 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:38.166177 (+ 1271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:38.166182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:38.166206 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:38.166340 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:38.166348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.167911 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.167917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:38.169235 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:38.169246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.170703 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.170706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:38.170791 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.170812 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 02:15:38.170858 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 02:15:38.170926 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":4}
I20260808 02:15:38.171680 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.165111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54918 (local address 127.18.184.131:42337)
0808 02:15:38.165619 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:38.165632 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:38.165663 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:15:38.165753 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:38.165758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:38.165988 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:38.166187 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:38.166498 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.167741 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.169568 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.170563 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.171254 (+ 691us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.171346 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:38.171387 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 02:15:38.171501 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":138,"threads_started":1}
I20260808 02:15:38.172791 2215 tablet_bootstrap.cc:492] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c: No bootstrap required, opened a new log
I20260808 02:15:38.172894 2215 ts_tablet_manager.cc:1397] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260808 02:15:38.173451 2215 raft_consensus.cc:348] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:38.173580 2215 raft_consensus.cc:374] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:38.173624 2215 raft_consensus.cc:729] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6831a9d158414ab192181b4aa3223d0c, State: Initialized, Role: FOLLOWER
I20260808 02:15:38.173772 2215 consensus_queue.cc:260] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:38.174077 2215 ts_tablet_manager.cc:1428] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:15:38.174300 2553 tablet_service.cc:1467] Processing CreateTablet for tablet 45572eea5e5f4002821e5f0da00e6f56 (DEFAULT_TABLE table=test-table [id=2b827c1fbc9643848a2efee0d302526d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:38.174352 2408 tablet_service.cc:1467] Processing CreateTablet for tablet 45572eea5e5f4002821e5f0da00e6f56 (DEFAULT_TABLE table=test-table [id=2b827c1fbc9643848a2efee0d302526d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:38.175333 2553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45572eea5e5f4002821e5f0da00e6f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:38.175467 2408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45572eea5e5f4002821e5f0da00e6f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:38.182464 2639 tablet_bootstrap.cc:492] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc: Bootstrap starting.
I20260808 02:15:38.182637 2640 tablet_bootstrap.cc:492] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: Bootstrap starting.
I20260808 02:15:38.184891 2639 tablet_bootstrap.cc:654] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:38.185398 2640 tablet_bootstrap.cc:654] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:38.186224 2640 log.cc:824] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:38.186225 2639 log.cc:824] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:38.188292 2640 tablet_bootstrap.cc:492] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: No bootstrap required, opened a new log
I20260808 02:15:38.188309 2639 tablet_bootstrap.cc:492] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc: No bootstrap required, opened a new log
I20260808 02:15:38.188513 2640 ts_tablet_manager.cc:1397] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 02:15:38.188526 2639 ts_tablet_manager.cc:1397] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:15:38.191885 2640 raft_consensus.cc:348] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:38.191921 2639 raft_consensus.cc:348] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:38.192145 2640 raft_consensus.cc:374] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:38.192139 2639 raft_consensus.cc:374] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:38.192215 2639 raft_consensus.cc:729] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e54dd0d65694a849e0f3b1f16bc43cc, State: Initialized, Role: FOLLOWER
I20260808 02:15:38.192235 2640 raft_consensus.cc:729] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a414042277e4eb6a6a11fcccfb98e75, State: Initialized, Role: FOLLOWER
I20260808 02:15:38.192691 2640 consensus_queue.cc:260] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:38.192691 2639 consensus_queue.cc:260] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:38.193485 2625 heartbeater.cc:503] Master 127.18.184.190:34007 was elected leader, sending a full tablet report...
I20260808 02:15:38.193485 2478 heartbeater.cc:503] Master 127.18.184.190:34007 was elected leader, sending a full tablet report...
I20260808 02:15:38.193840 2640 ts_tablet_manager.cc:1428] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260808 02:15:38.193838 2639 ts_tablet_manager.cc:1428] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260808 02:15:38.207154 2330 tablet.cc:2369] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:15:38.363267 2645 raft_consensus.cc:482] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:38.363494 2645 raft_consensus.cc:504] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:38.364802 2645 leader_election.cc:288] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831), 6831a9d158414ab192181b4aa3223d0c (127.18.184.129:40941)
I20260808 02:15:38.368872 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.365173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:38831 (local address 127.18.184.131:47673)
0808 02:15:38.365322 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:38.365346 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:38.365473 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:38.365735 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:38.365739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:38.365768 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:38.365919 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:38.365928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.366654 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.366658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:38.367633 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:38.367644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.368324 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.368329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:38.368537 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.368579 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 02:15:38.368669 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 02:15:38.368747 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":34}
I20260808 02:15:38.368918 2635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.365314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:47673 (local address 127.18.184.130:38831)
0808 02:15:38.365477 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:38.365482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:38.365501 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:15:38.365565 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:38.365567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:38.365622 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:38.365706 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:38.366061 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.366563 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.367791 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.368199 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.368296 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.368339 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:38.368705 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 02:15:38.368786 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:15:38.370054 2646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.365402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:60823 (local address 127.18.184.129:40941)
0808 02:15:38.365739 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:38.365744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:38.366060 (+ 316us) server_negotiation.cc:407] Connection header received
0808 02:15:38.366284 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:38.366287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:38.366363 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:38.366492 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:38.366829 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.367310 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.368686 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.369451 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.369562 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.369682 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:38.369885 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 02:15:38.369949 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":157,"threads_started":1}
I20260808 02:15:38.369987 2431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45572eea5e5f4002821e5f0da00e6f56" candidate_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a414042277e4eb6a6a11fcccfb98e75" is_pre_election: true
I20260808 02:15:38.370124 2647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.365188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:40941 (local address 127.18.184.131:60823)
0808 02:15:38.365956 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:38.365979 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:38.366156 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:38.366501 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:38.366504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:38.366521 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:38.366674 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:38.366682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.367429 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.367433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:38.368510 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:38.368521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.369612 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.369617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:38.369731 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.369762 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:15:38.369871 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 02:15:38.369940 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":423,"thread_start_us":117,"threads_started":1}
I20260808 02:15:38.370369 2431 raft_consensus.cc:2454] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e54dd0d65694a849e0f3b1f16bc43cc in term 0.
I20260808 02:15:38.370965 2503 leader_election.cc:302] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a414042277e4eb6a6a11fcccfb98e75, 5e54dd0d65694a849e0f3b1f16bc43cc; no voters:
I20260808 02:15:38.370959 2280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45572eea5e5f4002821e5f0da00e6f56" candidate_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6831a9d158414ab192181b4aa3223d0c" is_pre_election: true
I20260808 02:15:38.371279 2280 raft_consensus.cc:2454] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e54dd0d65694a849e0f3b1f16bc43cc in term 0.
I20260808 02:15:38.371317 2645 raft_consensus.cc:2781] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:15:38.371452 2645 raft_consensus.cc:482] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:15:38.371531 2645 raft_consensus.cc:3037] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 0 FOLLOWER]: Advancing to term 1
W20260808 02:15:38.372402 2626 tablet.cc:2369] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:15:38.372947 2645 raft_consensus.cc:504] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:38.373339 2645 leader_election.cc:288] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [CANDIDATE]: Term 1 election: Requested vote from peers 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831), 6831a9d158414ab192181b4aa3223d0c (127.18.184.129:40941)
I20260808 02:15:38.373724 2431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45572eea5e5f4002821e5f0da00e6f56" candidate_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a414042277e4eb6a6a11fcccfb98e75"
I20260808 02:15:38.373747 2280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45572eea5e5f4002821e5f0da00e6f56" candidate_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6831a9d158414ab192181b4aa3223d0c"
I20260808 02:15:38.373874 2431 raft_consensus.cc:3037] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:38.373884 2280 raft_consensus.cc:3037] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:38.374926 2280 raft_consensus.cc:2454] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e54dd0d65694a849e0f3b1f16bc43cc in term 1.
I20260808 02:15:38.374939 2431 raft_consensus.cc:2454] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e54dd0d65694a849e0f3b1f16bc43cc in term 1.
I20260808 02:15:38.375216 2502 leader_election.cc:302] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e54dd0d65694a849e0f3b1f16bc43cc, 6831a9d158414ab192181b4aa3223d0c; no voters:
I20260808 02:15:38.375417 2645 raft_consensus.cc:2781] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:38.375630 2645 raft_consensus.cc:686] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 1 LEADER]: Becoming Leader. State: Replica: 5e54dd0d65694a849e0f3b1f16bc43cc, State: Running, Role: LEADER
I20260808 02:15:38.375958 2645 consensus_queue.cc:237] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:38.379655 2125 catalog_manager.cc:5495] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131). New cstate: current_term: 1 leader_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } health_report { overall_health: UNKNOWN } } }
W20260808 02:15:38.412853 2479 tablet.cc:2369] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:15:38.415555 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.409741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54924 (local address 127.18.184.131:42337)
0808 02:15:38.409866 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:38.409871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:38.409985 (+ 114us) server_negotiation.cc:407] Connection header received
0808 02:15:38.410221 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:38.410226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:38.410321 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:38.410440 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:15:38.410711 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.411625 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.413957 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:38.414253 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:38.414375 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:38.414733 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:15:38.415131 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:15:38.415280 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:38.415322 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 02:15:38.415405 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 02:15:38.422506 2280 raft_consensus.cc:1261] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Refusing update from remote peer 5e54dd0d65694a849e0f3b1f16bc43cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:38.422509 2431 raft_consensus.cc:1261] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75 [term 1 FOLLOWER]: Refusing update from remote peer 5e54dd0d65694a849e0f3b1f16bc43cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:38.423283 2648 consensus_queue.cc:1035] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:38.423480 2645 consensus_queue.cc:1035] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:38.433840 2654 mvcc.cc:205] Tried to move back new op lower bound from 7316092266170998784 to 7316092265990729728. Current Snapshot: MvccSnapshot[applied={T|T < 7316092266170998784}]
I20260808 02:15:38.434477 2651 mvcc.cc:205] Tried to move back new op lower bound from 7316092266170998784 to 7316092265990729728. Current Snapshot: MvccSnapshot[applied={T|T < 7316092266170998784}]
I20260808 02:15:38.435674 2653 mvcc.cc:205] Tried to move back new op lower bound from 7316092266170998784 to 7316092265990729728. Current Snapshot: MvccSnapshot[applied={T|T < 7316092266170998784}]
I20260808 02:15:38.445556 2126 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:38.445773 2126 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:38.448432 2126 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Sending DeleteTablet for 3 replicas of tablet 45572eea5e5f4002821e5f0da00e6f56
I20260808 02:15:38.449369 2553 tablet_service.cc:1514] Processing DeleteTablet for tablet 45572eea5e5f4002821e5f0da00e6f56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:38 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:54918
I20260808 02:15:38.449575 2408 tablet_service.cc:1514] Processing DeleteTablet for tablet 45572eea5e5f4002821e5f0da00e6f56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:38 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:44366
I20260808 02:15:38.449947 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 2166
I20260808 02:15:38.450142 2259 tablet_service.cc:1514] Processing DeleteTablet for tablet 45572eea5e5f4002821e5f0da00e6f56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:38 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:40344
I20260808 02:15:38.457405 2661 tablet_replica.cc:330] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc: stopping tablet replica
I20260808 02:15:38.457482 2662 tablet_replica.cc:330] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: stopping tablet replica
I20260808 02:15:38.457938 2661 raft_consensus.cc:2229] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:15:38.457948 2662 raft_consensus.cc:2229] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:38.458366 2662 raft_consensus.cc:2258] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:38.458540 2661 raft_consensus.cc:2258] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:38.460305 2661 ts_tablet_manager.cc:1905] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:38.460315 2662 ts_tablet_manager.cc:1905] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 02:15:38.461850 2110 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:40941 (error 108)
I20260808 02:15:38.463218 2661 ts_tablet_manager.cc:1918] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:38.463761 2662 ts_tablet_manager.cc:1918] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:38.463929 2662 log.cc:1196] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/45572eea5e5f4002821e5f0da00e6f56
I20260808 02:15:38.463953 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 2334
I20260808 02:15:38.463999 2336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.463635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:40941 (local address 127.0.0.1:40352)
0808 02:15:38.463802 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:38.463886 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 02:15:38.467664 2110 catalog_manager.cc:4584] TS 6831a9d158414ab192181b4aa3223d0c (127.18.184.129:40941): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 45572eea5e5f4002821e5f0da00e6f56: Network error: Client connection negotiation failed: client connection to 127.18.184.129:40941: connect: Connection refused (error 111)
I20260808 02:15:38.471231 2661 log.cc:1196] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/45572eea5e5f4002821e5f0da00e6f56
I20260808 02:15:38.471671 2661 ts_tablet_manager.cc:1939] T 45572eea5e5f4002821e5f0da00e6f56 P 5e54dd0d65694a849e0f3b1f16bc43cc: Deleting consensus metadata
I20260808 02:15:38.473079 2110 catalog_manager.cc:4841] TS 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337): tablet 45572eea5e5f4002821e5f0da00e6f56 (table test-table [id=2b827c1fbc9643848a2efee0d302526d]) successfully deleted
I20260808 02:15:38.483018 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 2481
I20260808 02:15:38.484359 2336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.483311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:38831 (local address 127.0.0.1:44374)
0808 02:15:38.484148 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:38.484226 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.130:38831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":716}
I20260808 02:15:38.490806 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 2091
I20260808 02:15:38.499716 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34007
--webserver_interface=127.18.184.190
--webserver_port=41775
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:34007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:38.613313 2665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:38.613605 2665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:38.613646 2665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:38.617664 2665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:38.617733 2665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:38.617753 2665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:38.617769 2665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:38.617785 2665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:38.623275 2665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:34007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:34007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=41775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:38.624496 2665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:38.625651 2665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:38.634851 2670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:38.635058 2665 server_base.cc:1034] running on GCE node
W20260808 02:15:38.634852 2671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:38.634887 2673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:38.635968 2665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:38.637337 2665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:38.638549 2665 hybrid_clock.cc:648] HybridClock initialized: now 1786155338638536 us; error 36 us; skew 500 ppm
Aug 08 02:15:38 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155338, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:38.641476 2665 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:15:38.642408 2665 webserver.cc:460] Webserver started at http://127.18.184.190:41775/ using document root <none> and password file <none>
I20260808 02:15:38.642964 2665 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:38.643025 2665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:38.647274 2665 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:15:38.649533 2680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:38.650879 2665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:38.651031 2665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "b1e78155df354d9f8b986cf60daf5d2e"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "6b17bdeb27d7b3c5d96ebf390a33810c"
server_key_iv: "ba2621a9dd1dc272636dff7148590c91"
server_key_version: "encryptionkey@0"
I20260808 02:15:38.651615 2665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:38.684827 2665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:38.688004 2665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:38.688181 2665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:38.695654 2665 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:34007
I20260808 02:15:38.695943 2737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:34007 every 8 connection(s)
I20260808 02:15:38.696887 2665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:38.697229 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 2665
I20260808 02:15:38.697809 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:40941
--local_ip_for_outbound_sockets=127.18.184.129
--tserver_master_addrs=127.18.184.190:34007
--webserver_port=45943
--webserver_interface=127.18.184.129
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:38.702687 2739 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:15:38.704523 2739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Bootstrap starting.
I20260808 02:15:38.723580 2739 log.cc:824] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:38.734910 2739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:15:38.735340 2739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e: Bootstrap complete.
I20260808 02:15:38.739585 2739 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } }
I20260808 02:15:38.739997 2739 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1e78155df354d9f8b986cf60daf5d2e, State: Initialized, Role: FOLLOWER
I20260808 02:15:38.740541 2739 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } }
I20260808 02:15:38.740660 2739 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:38.740744 2739 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:38.740870 2739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 2 FOLLOWER]: Advancing to term 3
I20260808 02:15:38.742862 2739 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } }
I20260808 02:15:38.743201 2739 leader_election.cc:302] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1e78155df354d9f8b986cf60daf5d2e; no voters:
I20260808 02:15:38.743575 2739 leader_election.cc:288] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 02:15:38.743690 2744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 3 FOLLOWER]: Leader election won for term 3
I20260808 02:15:38.743883 2744 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [term 3 LEADER]: Becoming Leader. State: Replica: b1e78155df354d9f8b986cf60daf5d2e, State: Running, Role: LEADER
I20260808 02:15:38.744240 2744 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } }
I20260808 02:15:38.744894 2739 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:38.745714 2745 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b1e78155df354d9f8b986cf60daf5d2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } } }
I20260808 02:15:38.745857 2745 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:38.746600 2746 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1e78155df354d9f8b986cf60daf5d2e. Latest consensus state: current_term: 3 leader_uuid: "b1e78155df354d9f8b986cf60daf5d2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e78155df354d9f8b986cf60daf5d2e" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 34007 } } }
I20260808 02:15:38.746702 2746 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e78155df354d9f8b986cf60daf5d2e [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:38.747252 2752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:38.751714 2752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:38.752269 2752 catalog_manager.cc:1259] Loaded cluster ID: 8a56baa8969249a090122ad88dd96f4f
I20260808 02:15:38.752362 2752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 02:15:38.753367 2752 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5e7e0539ad google::LogMessage::Fail() at ??:0
@ 0x7f5e7e05688a google::LogMessage::SendToLog() at ??:0
@ 0x7f5e7e0534de google::LogMessage::Flush() at ??:0
@ 0x7f5e7e054a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5e88996fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f5e88997d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5e88994579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f5e889d8640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5e802a53d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5e802a5c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f5e802a73f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5e80296512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5e818da6db start_thread at ??:0
@ 0x7f5e7cb5161f clone at ??:0
W20260808 02:15:38.848255 2742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:38.848537 2742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:38.848587 2742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:38.852818 2742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:38.852887 2742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:38.852950 2742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:15:38.858662 2742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:40941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=45943
--tserver_master_addrs=127.18.184.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:38.859844 2742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:38.860901 2742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:38.871205 2785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:38.871264 2787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:38.871332 2742 server_base.cc:1034] running on GCE node
W20260808 02:15:38.871205 2784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:38.871927 2742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:38.872813 2742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:38.874008 2742 hybrid_clock.cc:648] HybridClock initialized: now 1786155338873948 us; error 65 us; skew 500 ppm
Aug 08 02:15:38 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155338, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:38.877871 2742 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:15:38.878928 2742 webserver.cc:460] Webserver started at http://127.18.184.129:45943/ using document root <none> and password file <none>
I20260808 02:15:38.879650 2742 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:38.879724 2742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:38.884068 2742 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:15:38.886705 2794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:38.888124 2742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:15:38.888293 2742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "6831a9d158414ab192181b4aa3223d0c"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "bab146268cb40b7f4ca5f01d4ba0d89b"
server_key_iv: "a11c24e2062d17add38555ae31260a44"
server_key_version: "encryptionkey@0"
I20260808 02:15:38.888907 2742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:38.921988 2742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:38.926666 2742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:38.926903 2742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:38.927747 2742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:38.929289 2802 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:15:38.935344 2742 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:15:38.935483 2742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260808 02:15:38.935571 2742 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:15:38.937516 2742 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:15:38.937594 2742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260808 02:15:38.937927 2802 tablet_bootstrap.cc:492] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c: Bootstrap starting.
I20260808 02:15:38.948062 2742 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:40941
I20260808 02:15:38.948249 2916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:40941 every 8 connection(s)
I20260808 02:15:38.949210 2742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:15:38.957355 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 2742
I20260808 02:15:38.957870 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:38831
--local_ip_for_outbound_sockets=127.18.184.130
--tserver_master_addrs=127.18.184.190:34007
--webserver_port=36331
--webserver_interface=127.18.184.130
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:38.963101 2802 log.cc:824] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:38.973359 2802 tablet_bootstrap.cc:492] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:15:38.973848 2802 tablet_bootstrap.cc:492] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c: Bootstrap complete.
I20260808 02:15:38.975553 2802 ts_tablet_manager.cc:1397] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c: Time spent bootstrapping tablet: real 0.038s user 0.023s sys 0.011s
I20260808 02:15:38.979535 2802 raft_consensus.cc:348] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:38.980120 2802 raft_consensus.cc:729] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6831a9d158414ab192181b4aa3223d0c, State: Initialized, Role: FOLLOWER
I20260808 02:15:38.980687 2802 consensus_queue.cc:260] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:38.981889 2802 ts_tablet_manager.cc:1428] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
W20260808 02:15:39.008971 2919 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:15:38.951292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:54627)
0808 02:15:38.951789 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:38.951846 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:38.952664 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:39.008067 (+ 55403us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":149,"threads_started":1}
W20260808 02:15:39.009697 2917 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:34007 (0 consecutive failures): Network error: Failed to ping master at 127.18.184.190:34007: Client connection negotiation failed: client connection to 127.18.184.190:34007: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 02:15:39.010593 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:39.010291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:60537)
0808 02:15:39.010449 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:39.010519 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 02:15:39.011370 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:39.011164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:56171)
0808 02:15:39.011262 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:39.011305 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 02:15:39.011613 2917 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:15:39.084815 2920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:39.085127 2920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:39.085193 2920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:39.089439 2920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:39.089521 2920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:39.089620 2920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:15:39.095498 2920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:38831
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=36331
--tserver_master_addrs=127.18.184.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:39.096675 2920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:39.097816 2920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:39.108569 2928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:39.108573 2931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:39.108924 2920 server_base.cc:1034] running on GCE node
W20260808 02:15:39.109048 2929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:39.109604 2920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:39.110319 2920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:39.111550 2920 hybrid_clock.cc:648] HybridClock initialized: now 1786155339111528 us; error 33 us; skew 500 ppm
Aug 08 02:15:39 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155339, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:39.115061 2920 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:15:39.116158 2920 webserver.cc:460] Webserver started at http://127.18.184.130:36331/ using document root <none> and password file <none>
I20260808 02:15:39.116773 2920 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:39.116875 2920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:39.121345 2920 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:15:39.123817 2938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:39.125319 2920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:39.125460 2920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "2a414042277e4eb6a6a11fcccfb98e75"
format_stamp: "Formatted at 2026-08-08 02:15:36 on dist-test-slave-9gcw"
server_key: "6daf6a5a96af2fc1cbb40d27e4de924d"
server_key_iv: "30b3c2d49144987aa26d0d04a1fd1812"
server_key_version: "encryptionkey@0"
I20260808 02:15:39.126086 2920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:39.150297 2920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:39.154570 2920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:39.154987 2920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:39.155730 2920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:39.156975 2945 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260808 02:15:39.160997 2945 ts_tablet_manager.cc:1852] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260808 02:15:39.161486 2945 ts_tablet_manager.cc:1905] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:39.166043 2945 ts_tablet_manager.cc:1918] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:39.166254 2945 log.cc:1196] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/45572eea5e5f4002821e5f0da00e6f56
I20260808 02:15:39.166679 2945 ts_tablet_manager.cc:1939] T 45572eea5e5f4002821e5f0da00e6f56 P 2a414042277e4eb6a6a11fcccfb98e75: Deleting consensus metadata
I20260808 02:15:39.167060 2920 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260808 02:15:39.167124 2920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.000s
I20260808 02:15:39.167212 2920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:39.167254 2920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:39.177258 2920 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:38831
I20260808 02:15:39.177374 3056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:38831 every 8 connection(s)
I20260808 02:15:39.178210 2920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:15:39.182037 3059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:39.180543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:44277)
0808 02:15:39.181086 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:39.181275 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":168,"threads_started":1}
W20260808 02:15:39.182600 3057 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:34007 (0 consecutive failures): Network error: Failed to ping master at 127.18.184.190:34007: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
I20260808 02:15:39.183339 3059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:39.183089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:45093)
0808 02:15:39.183223 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:39.183276 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 02:15:39.183974 3059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:39.183752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:59385)
0808 02:15:39.183874 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:39.183922 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 02:15:39.184144 3057 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:15:39.186762 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 2920
I20260808 02:15:39.187232 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:42337
--local_ip_for_outbound_sockets=127.18.184.131
--tserver_master_addrs=127.18.184.190:34007
--webserver_port=39359
--webserver_interface=127.18.184.131
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:39.310635 3060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:39.310945 3060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:39.310994 3060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:39.315366 3060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:39.315450 3060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:39.315522 3060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:15:39.321282 3060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:42337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=39359
--tserver_master_addrs=127.18.184.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:39.322489 3060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:39.323589 3060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:39.334528 3066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:39.334563 3065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:39.334563 3068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:39.335206 3060 server_base.cc:1034] running on GCE node
I20260808 02:15:39.335628 3060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:39.336246 3060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:39.337443 3060 hybrid_clock.cc:648] HybridClock initialized: now 1786155339337431 us; error 31 us; skew 500 ppm
Aug 08 02:15:39 dist-test-slave-9gcw krb5kdc[1501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155339, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:39.340379 3060 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:15:39.341423 3060 webserver.cc:460] Webserver started at http://127.18.184.131:39359/ using document root <none> and password file <none>
I20260808 02:15:39.342058 3060 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:39.342162 3060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:39.346544 3060 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:15:39.348810 3075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:39.350162 3060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 02:15:39.350353 3060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "5e54dd0d65694a849e0f3b1f16bc43cc"
format_stamp: "Formatted at 2026-08-08 02:15:37 on dist-test-slave-9gcw"
server_key: "5dcabd45d00ace439bbe97c2b7cf944e"
server_key_iv: "fcece7348cb29f98ffddec70e2f67a2f"
server_key_version: "encryptionkey@0"
I20260808 02:15:39.351017 3060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:39.376842 3060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:39.380818 3060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:39.381088 3060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:39.381843 3060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:39.382779 3060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:39.382833 3060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:39.382879 3060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:39.382897 3060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:39.393505 3060 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:42337
I20260808 02:15:39.393532 3199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:42337 every 8 connection(s)
I20260808 02:15:39.394630 3060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:15:39.395895 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 3060
I20260808 02:15:39.399359 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:39.397424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:52005)
0808 02:15:39.397900 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:39.398021 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":115,"threads_started":1}
W20260808 02:15:39.399979 3202 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:34007 (0 consecutive failures): Network error: Failed to ping master at 127.18.184.190:34007: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
I20260808 02:15:39.400702 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:39.400444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:47317)
0808 02:15:39.400576 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:39.400639 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 02:15:39.401427 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:39.401211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:39571)
0808 02:15:39.401312 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:39.401371 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 02:15:39.401661 3202 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:15:40.012758 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:40.012294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:33497)
0808 02:15:40.012586 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:40.012651 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":79,"threads_started":1}
I20260808 02:15:40.135272 3207 raft_consensus.cc:482] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:40.135522 3207 raft_consensus.cc:504] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:40.136686 3207 leader_election.cc:288] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337), 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831)
Aug 08 02:15:40 dist-test-slave-9gcw krb5kdc[1501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155338, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.131@KRBTEST.COM
Aug 08 02:15:40 dist-test-slave-9gcw krb5kdc[1501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155338, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.130@KRBTEST.COM
I20260808 02:15:40.154419 3210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:40.137200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:56889 (local address 127.18.184.131:42337)
0808 02:15:40.137633 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:40.137650 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:40.137693 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:15:40.138577 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:40.138588 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:40.139069 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:40.139415 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:40.141353 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:40.142989 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:40.145000 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:40.145739 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:40.145917 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:40.149658 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:40.149667 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:40.149683 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:40.149718 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:40.151698 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:40.152295 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:40.152301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:40.152306 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:40.152427 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:40.152826 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:40.152834 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:40.152835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:40.153267 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:40.153409 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:40.153965 (+ 556us) server_negotiation.cc:299] Negotiation successful
0808 02:15:40.154228 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":109,"threads_started":1}
I20260808 02:15:40.154407 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:40.136975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:42337 (local address 127.18.184.129:56889)
0808 02:15:40.137230 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:40.137248 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:40.137390 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:40.139535 (+ 2145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:40.139542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:40.140228 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:40.141100 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:40.141122 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:40.143174 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:40.143177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:40.144861 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:40.144874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:40.145887 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:40.145891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:40.146095 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:40.146903 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:40.146930 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:40.149373 (+ 2443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:40.151829 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:40.151835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:40.151837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:40.152166 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:40.152530 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:40.152535 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:40.152536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:40.152683 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:40.153385 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:40.153391 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:40.153636 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 02:15:40.153827 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 02:15:40.154196 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":419,"spinlock_wait_cycles":28928}
I20260808 02:15:40.154433 3209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:40.137164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:57899 (local address 127.18.184.130:38831)
0808 02:15:40.137633 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:40.137650 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:40.137694 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:15:40.138611 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:40.138618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:40.139069 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:40.139415 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:40.141383 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:40.142989 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:40.145000 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:40.145739 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:40.145917 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:40.149630 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:40.149635 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:40.149649 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:40.149679 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:40.151648 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:40.152296 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:40.152301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:40.152305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:40.152422 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:40.152790 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:40.152793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:40.152794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:40.153257 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:40.153410 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:40.153964 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 02:15:40.154253 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":88,"threads_started":1}
I20260808 02:15:40.154455 3208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:40.136975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:38831 (local address 127.18.184.129:57899)
0808 02:15:40.137455 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:40.137469 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:40.137576 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:40.139534 (+ 1958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:40.139542 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:40.140228 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:40.141100 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:40.141122 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:40.143174 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:40.143177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:40.144861 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:40.144874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:40.146017 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:40.146023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:40.146104 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:40.146903 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:40.146938 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:40.149376 (+ 2438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:40.151790 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:40.151797 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:40.151811 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:40.152163 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:40.152524 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:40.152528 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:40.152531 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:40.152680 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:40.153384 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:40.153390 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:40.153636 (+ 246us) client_negotiation.cc:769] Sending connection context
0808 02:15:40.153828 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 02:15:40.154282 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":525,"spinlock_wait_cycles":537344,"thread_start_us":78,"threads_started":1}
W20260808 02:15:40.157220 2799 leader_election.cc:341] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831): Not found: Tablet not found: 45572eea5e5f4002821e5f0da00e6f56
W20260808 02:15:40.157408 2798 leader_election.cc:341] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337): Not found: Tablet not found: 45572eea5e5f4002821e5f0da00e6f56
I20260808 02:15:40.157497 2798 leader_election.cc:302] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6831a9d158414ab192181b4aa3223d0c; no voters: 2a414042277e4eb6a6a11fcccfb98e75, 5e54dd0d65694a849e0f3b1f16bc43cc
I20260808 02:15:40.157739 3207 raft_consensus.cc:2726] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:15:40.185075 3211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:40.184675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:51933)
0808 02:15:40.184937 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:40.184989 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":69,"threads_started":1}
I20260808 02:15:40.402786 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:40.402245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:50125)
0808 02:15:40.402564 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:40.402625 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":124,"threads_started":1}
I20260808 02:15:41.014246 3213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:41.013729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:39961)
0808 02:15:41.014091 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:41.014147 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":83,"threads_started":1}
I20260808 02:15:41.186409 3214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:41.185953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:33947)
0808 02:15:41.186238 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:41.186304 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":74,"threads_started":1}
I20260808 02:15:41.404382 3215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:41.403853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:46567)
0808 02:15:41.404166 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:41.404252 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":86,"threads_started":1}
I20260808 02:15:42.015518 3216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:42.015038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:38499)
0808 02:15:42.015334 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:42.015413 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":92,"threads_started":1}
I20260808 02:15:42.187638 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:42.187174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:39097)
0808 02:15:42.187480 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:42.187539 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":113,"threads_started":1}
I20260808 02:15:42.261639 3218 raft_consensus.cc:482] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:42.261869 3218 raft_consensus.cc:504] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:42.262320 3218 leader_election.cc:288] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337), 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831)
W20260808 02:15:42.263113 2798 leader_election.cc:341] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337): Not found: Tablet not found: 45572eea5e5f4002821e5f0da00e6f56
W20260808 02:15:42.263214 2799 leader_election.cc:341] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831): Not found: Tablet not found: 45572eea5e5f4002821e5f0da00e6f56
I20260808 02:15:42.263340 2799 leader_election.cc:302] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6831a9d158414ab192181b4aa3223d0c; no voters: 2a414042277e4eb6a6a11fcccfb98e75, 5e54dd0d65694a849e0f3b1f16bc43cc
I20260808 02:15:42.263540 3218 raft_consensus.cc:2726] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:15:42.405869 3219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:42.405308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:47191)
0808 02:15:42.405661 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:42.405725 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":118,"threads_started":1}
I20260808 02:15:43.016858 3220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:43.016345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:47187)
0808 02:15:43.016621 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:43.016749 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":83,"threads_started":1}
I20260808 02:15:43.189080 3221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:43.188601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:60357)
0808 02:15:43.188912 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:43.188978 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":91,"threads_started":1}
I20260808 02:15:43.407298 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:43.406734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:45893)
0808 02:15:43.407051 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:43.407166 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":105,"threads_started":1}
I20260808 02:15:44.018363 3223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.017885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:49817)
0808 02:15:44.018185 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:44.018240 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":77,"threads_started":1}
I20260808 02:15:44.190557 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.190073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:43861)
0808 02:15:44.190385 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:44.190453 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":89,"threads_started":1}
I20260808 02:15:44.408581 3225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.408084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:42901)
0808 02:15:44.408380 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:44.408449 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":101,"threads_started":1}
I20260808 02:15:44.759131 3226 raft_consensus.cc:482] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:44.759284 3226 raft_consensus.cc:504] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:44.759938 3226 leader_election.cc:288] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337), 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831)
W20260808 02:15:44.760936 2799 leader_election.cc:341] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831): Not found: Tablet not found: 45572eea5e5f4002821e5f0da00e6f56
W20260808 02:15:44.761106 2798 leader_election.cc:341] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337): Not found: Tablet not found: 45572eea5e5f4002821e5f0da00e6f56
I20260808 02:15:44.761250 2798 leader_election.cc:302] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6831a9d158414ab192181b4aa3223d0c; no voters: 2a414042277e4eb6a6a11fcccfb98e75, 5e54dd0d65694a849e0f3b1f16bc43cc
I20260808 02:15:44.761462 3226 raft_consensus.cc:2726] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:15:45.019790 3227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:45.019190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:59161)
0808 02:15:45.019579 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:45.019662 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":115,"threads_started":1}
I20260808 02:15:45.192030 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:45.191473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:40269)
0808 02:15:45.191835 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:45.191909 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":102,"threads_started":1}
I20260808 02:15:45.410006 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:45.409518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:45787)
0808 02:15:45.409842 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:45.409910 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":108,"threads_started":1}
I20260808 02:15:46.021137 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:46.020666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:50637)
0808 02:15:46.020983 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:46.021038 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":95,"threads_started":1}
I20260808 02:15:46.193730 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:46.193040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:52169)
0808 02:15:46.193488 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:46.193555 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":166,"threads_started":1}
I20260808 02:15:46.411299 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:46.410809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:39191)
0808 02:15:46.411133 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:46.411189 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":94,"threads_started":1}
I20260808 02:15:47.022404 3234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:47.021973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:46423)
0808 02:15:47.022234 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:47.022292 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
I20260808 02:15:47.149130 3235 raft_consensus.cc:482] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:47.149338 3235 raft_consensus.cc:504] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:47.149788 3235 leader_election.cc:288] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337), 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831)
W20260808 02:15:47.150686 2799 leader_election.cc:341] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831): Not found: Tablet not found: 45572eea5e5f4002821e5f0da00e6f56
W20260808 02:15:47.150787 2798 leader_election.cc:341] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337): Not found: Tablet not found: 45572eea5e5f4002821e5f0da00e6f56
I20260808 02:15:47.150864 2798 leader_election.cc:302] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6831a9d158414ab192181b4aa3223d0c; no voters: 2a414042277e4eb6a6a11fcccfb98e75, 5e54dd0d65694a849e0f3b1f16bc43cc
I20260808 02:15:47.151012 3235 raft_consensus.cc:2726] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:15:47.195003 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:47.194546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:46221)
0808 02:15:47.194846 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:47.194904 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":129,"threads_started":1}
I20260808 02:15:47.412623 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:47.412164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:57673)
0808 02:15:47.412464 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:47.412526 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":99,"threads_started":1}
I20260808 02:15:48.023831 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:48.023295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:36323)
0808 02:15:48.023654 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:48.023728 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":100,"threads_started":1}
I20260808 02:15:48.196233 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:48.195774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:55269)
0808 02:15:48.196068 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:48.196129 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":102,"threads_started":1}
I20260808 02:15:48.414042 3240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:48.413503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:36897)
0808 02:15:48.413835 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:48.413913 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":82,"threads_started":1}
I20260808 02:15:49.025496 3241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:49.024921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:42401)
0808 02:15:49.025323 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:49.025393 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":96,"threads_started":1}
I20260808 02:15:49.197681 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:49.197138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:43947)
0808 02:15:49.197510 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:49.197575 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":104,"threads_started":1}
I20260808 02:15:49.415535 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:49.415077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:48097)
0808 02:15:49.415362 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:49.415430 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
I20260808 02:15:50.027470 3244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:50.026685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:34097)
0808 02:15:50.027181 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:50.027303 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":139,"threads_started":1}
I20260808 02:15:50.199239 3245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:50.198687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:46275)
0808 02:15:50.199003 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:50.199074 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":102,"threads_started":1}
I20260808 02:15:50.417276 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:50.416396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:53039)
0808 02:15:50.416933 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:50.417018 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":164,"threads_started":1}
I20260808 02:15:51.030426 3247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:51.029375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:55125)
0808 02:15:51.030077 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:51.030266 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":135,"threads_started":1}
I20260808 02:15:51.200855 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:51.200332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:44321)
0808 02:15:51.200630 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:51.200699 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":83,"threads_started":1}
I20260808 02:15:51.418776 3249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:51.418238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:36385)
0808 02:15:51.418582 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:51.418667 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":95,"threads_started":1}
I20260808 02:15:52.031984 3250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:52.031466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:38509)
0808 02:15:52.031808 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:52.031880 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":100,"threads_started":1}
I20260808 02:15:52.202327 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:52.201797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:36925)
0808 02:15:52.202168 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:52.202223 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":142,"threads_started":1}
I20260808 02:15:52.420083 3252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:52.419653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:41785)
0808 02:15:52.419920 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:52.419982 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260808 02:15:52.519104 3254 raft_consensus.cc:482] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:52.519371 3254 raft_consensus.cc:504] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e54dd0d65694a849e0f3b1f16bc43cc" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42337 } } peers { permanent_uuid: "2a414042277e4eb6a6a11fcccfb98e75" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38831 } } peers { permanent_uuid: "6831a9d158414ab192181b4aa3223d0c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40941 } }
I20260808 02:15:52.519886 3254 leader_election.cc:288] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337), 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831)
W20260808 02:15:52.520797 2798 leader_election.cc:341] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e54dd0d65694a849e0f3b1f16bc43cc (127.18.184.131:42337): Not found: Tablet not found: 45572eea5e5f4002821e5f0da00e6f56
W20260808 02:15:52.520906 2799 leader_election.cc:341] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a414042277e4eb6a6a11fcccfb98e75 (127.18.184.130:38831): Not found: Tablet not found: 45572eea5e5f4002821e5f0da00e6f56
I20260808 02:15:52.520965 2799 leader_election.cc:302] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6831a9d158414ab192181b4aa3223d0c; no voters: 2a414042277e4eb6a6a11fcccfb98e75, 5e54dd0d65694a849e0f3b1f16bc43cc
I20260808 02:15:52.521108 3254 raft_consensus.cc:2726] T 45572eea5e5f4002821e5f0da00e6f56 P 6831a9d158414ab192181b4aa3223d0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:15:53.033650 3255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:53.032957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:35897)
0808 02:15:53.033436 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:53.033516 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":122,"threads_started":1}
I20260808 02:15:53.203639 3257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:53.203166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:47109)
0808 02:15:53.203471 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:53.203534 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":98,"threads_started":1}
I20260808 02:15:53.421711 3258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:53.421140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.131:35719)
0808 02:15:53.421538 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:53.421606 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":87,"threads_started":1}
I20260808 02:15:54.035207 3259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:54.034732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.129:57529)
0808 02:15:54.035039 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:54.035102 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":101,"threads_started":1}
I20260808 02:15:54.205402 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:54.204829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:34007 (local address 127.18.184.130:34853)
0808 02:15:54.205189 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:54.205294 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:34007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":99,"threads_started":1}
I20260808 02:15:54.396800 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 2742
I20260808 02:15:54.405541 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 2920
I20260808 02:15:54.413425 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 3060
2026-08-08T02:15:54Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19590 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:15:54 dist-test-slave-9gcw krb5kdc[3264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:15:54 dist-test-slave-9gcw krb5kdc[3264](info): set up 2 sockets
Aug 08 02:15:54 dist-test-slave-9gcw krb5kdc[3264](info): commencing operation
krb5kdc: starting...
W20260808 02:15:55.528820 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:15:55 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:15:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:15:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:15:55.674201 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:40287
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:40287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:55.799746 3278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:55.800081 3278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:55.800168 3278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:55.804917 3278 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 02:15:55.805006 3278 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 02:15:55.805033 3278 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 02:15:55.805058 3278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:55.805099 3278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:55.805167 3278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:55.805203 3278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:55.805238 3278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:55.814215 3278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:40287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:40287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:55.815778 3278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:55.816936 3278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:55.828724 3285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:55.828735 3288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:55.828717 3284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:55.829478 3278 server_base.cc:1034] running on GCE node
I20260808 02:15:55.830238 3278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:55.831951 3278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:55.833134 3278 hybrid_clock.cc:648] HybridClock initialized: now 1786155355833107 us; error 48 us; skew 500 ppm
Aug 08 02:15:55 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155355, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:55.837389 3278 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:15:55.838894 3278 webserver.cc:460] Webserver started at http://127.18.184.190:34609/ using document root <none> and password file <none>
I20260808 02:15:55.839752 3278 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:55.839860 3278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:55.840149 3278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:55.843422 3278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "a4228483587740a1a8333148c6b43e66"
format_stamp: "Formatted at 2026-08-08 02:15:55 on dist-test-slave-9gcw"
server_key: "7759432198d95601bc323e16a15151a5"
server_key_iv: "1f6f745e5d893c1fd826ebe4e00b2f76"
server_key_version: "encryptionkey@0"
I20260808 02:15:55.844448 3278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "a4228483587740a1a8333148c6b43e66"
format_stamp: "Formatted at 2026-08-08 02:15:55 on dist-test-slave-9gcw"
server_key: "7759432198d95601bc323e16a15151a5"
server_key_iv: "1f6f745e5d893c1fd826ebe4e00b2f76"
server_key_version: "encryptionkey@0"
I20260808 02:15:55.851159 3278 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260808 02:15:55.855078 3296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:55.856825 3278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:15:55.857103 3278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "a4228483587740a1a8333148c6b43e66"
format_stamp: "Formatted at 2026-08-08 02:15:55 on dist-test-slave-9gcw"
server_key: "7759432198d95601bc323e16a15151a5"
server_key_iv: "1f6f745e5d893c1fd826ebe4e00b2f76"
server_key_version: "encryptionkey@0"
I20260808 02:15:55.857331 3278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:55.893067 3278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:55.896612 3278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:55.896966 3278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:55.905426 3278 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:40287
I20260808 02:15:55.905726 3353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:40287 every 8 connection(s)
I20260808 02:15:55.906559 3278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:55.909415 3354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:55.913774 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 3278
I20260808 02:15:55.913889 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:15:55.914213 19170 external_mini_cluster.cc:1428] Setting key 5d73690bb2f37c2b9618143c8b7b7b8f
I20260808 02:15:55.915366 3354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Bootstrap starting.
I20260808 02:15:55.917742 3354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:55.918748 3354 log.cc:824] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:55.920945 3354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: No bootstrap required, opened a new log
Aug 08 02:15:55 dist-test-slave-9gcw krb5kdc[3264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:55.925791 3354 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } }
I20260808 02:15:55.926133 3354 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:55.926192 3354 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4228483587740a1a8333148c6b43e66, State: Initialized, Role: FOLLOWER
I20260808 02:15:55.926692 3354 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } }
I20260808 02:15:55.926872 3354 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:55.926937 3354 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:55.927084 3354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:55.928304 3354 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } }
I20260808 02:15:55.928676 3354 leader_election.cc:302] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4228483587740a1a8333148c6b43e66; no voters:
I20260808 02:15:55.929054 3354 leader_election.cc:288] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:15:55.929207 3360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:55.929555 3360 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 1 LEADER]: Becoming Leader. State: Replica: a4228483587740a1a8333148c6b43e66, State: Running, Role: LEADER
I20260808 02:15:55.929963 3360 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } }
I20260808 02:15:55.930267 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:55.915801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.18.184.190:40287)
0808 02:15:55.916255 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:55.916272 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:55.916310 (+ 38us) server_negotiation.cc:407] Connection header received
0808 02:15:55.916973 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:55.916989 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:55.917398 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:55.917680 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:55.918687 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:55.919798 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:55.920957 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:55.921686 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:55.921815 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:55.924679 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:55.924686 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:55.924704 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:55.924737 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:55.926884 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:55.927415 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:55.927421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:55.927427 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:55.927561 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:55.927850 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:55.927853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:55.927855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:55.928418 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:55.928645 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:55.928938 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 02:15:55.929288 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":147,"threads_started":1}
I20260808 02:15:55.930469 3354 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:55.931262 3360 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4228483587740a1a8333148c6b43e66. Latest consensus state: current_term: 1 leader_uuid: "a4228483587740a1a8333148c6b43e66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } } }
I20260808 02:15:55.931507 3360 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:55.931716 3361 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4228483587740a1a8333148c6b43e66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } } }
I20260808 02:15:55.931813 3361 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:55.932530 3369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:55.935714 3369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:55.942023 3369 catalog_manager.cc:1347] Generated new cluster ID: ad53c1bc050d4f99b6dde1ab60fee4bd
I20260808 02:15:55.942175 3369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:15:55.957448 3369 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:15:55.958508 3369 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:55.962580 3369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Generated new TSK 0
I20260808 02:15:55.963526 3369 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:15:56.004947 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40287
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:56.153192 3381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:56.153544 3381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:56.153649 3381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:56.157866 3381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:56.157960 3381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:56.158051 3381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:15:56.164120 3381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:56.165321 3381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:56.166390 3381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:56.176483 3388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:56.176503 3386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:56.176499 3390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:56.176709 3381 server_base.cc:1034] running on GCE node
I20260808 02:15:56.177358 3381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:56.178017 3381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:56.179174 3381 hybrid_clock.cc:648] HybridClock initialized: now 1786155356179151 us; error 36 us; skew 500 ppm
Aug 08 02:15:56 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155356, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:56.182031 3381 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:15:56.182935 3381 webserver.cc:460] Webserver started at http://127.18.184.129:39235/ using document root <none> and password file <none>
I20260808 02:15:56.183557 3381 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:56.183653 3381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:56.183904 3381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:56.186088 3381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "4b89f0cc6d3b4856998cc7d081545471"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "baba4c5d31da52385d14ddb4ec59ea6c"
server_key_iv: "96386123645954c31047f5c0386a110c"
server_key_version: "encryptionkey@0"
I20260808 02:15:56.186681 3381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "4b89f0cc6d3b4856998cc7d081545471"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "baba4c5d31da52385d14ddb4ec59ea6c"
server_key_iv: "96386123645954c31047f5c0386a110c"
server_key_version: "encryptionkey@0"
I20260808 02:15:56.190533 3381 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:15:56.193126 3398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:56.194504 3381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:56.194653 3381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "4b89f0cc6d3b4856998cc7d081545471"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "baba4c5d31da52385d14ddb4ec59ea6c"
server_key_iv: "96386123645954c31047f5c0386a110c"
server_key_version: "encryptionkey@0"
I20260808 02:15:56.194785 3381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:56.216780 3381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:56.220696 3381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:56.220983 3381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:56.221691 3381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:56.222580 3381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:56.222654 3381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:56.222730 3381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:56.222783 3381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:56.233296 3381 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:44709
I20260808 02:15:56.233325 3515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:44709 every 8 connection(s)
I20260808 02:15:56.234233 3381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:15:56.234876 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 3381
I20260808 02:15:56.235038 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:15:56.235345 19170 external_mini_cluster.cc:1428] Setting key 909066771bf07812773ef79ec673c046
Aug 08 02:15:56 dist-test-slave-9gcw krb5kdc[3264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155356, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
I20260808 02:15:56.251422 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.236231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:50827 (local address 127.18.184.190:40287)
0808 02:15:56.236379 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:56.236384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:56.237672 (+ 1288us) server_negotiation.cc:407] Connection header received
0808 02:15:56.239759 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:56.239762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:56.239824 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:56.239902 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:56.241365 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.241961 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.243917 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.244146 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.244225 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.247398 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:56.247403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:56.247405 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:56.247431 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:56.249021 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:56.249689 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:56.249699 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:56.249701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:56.249782 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:56.250197 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:56.250200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:56.250202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:56.250446 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:56.250554 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:56.251091 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 02:15:56.251233 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:15:56.252346 3518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.236668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:50827)
0808 02:15:56.237490 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:56.237558 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:56.239524 (+ 1966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:56.240078 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:56.240086 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:56.240464 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:56.241186 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:56.241210 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.242116 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.242123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:56.243762 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:56.243777 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.244330 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.244337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:56.244577 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.245377 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:56.245403 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:56.247191 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:56.249190 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:56.249197 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:56.249214 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:56.249569 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:56.249924 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:56.249927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:56.249929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:56.250101 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:56.250556 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:56.250564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:56.250837 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 02:15:56.251027 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 02:15:56.251384 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":128,"threads_started":1}
I20260808 02:15:56.253779 3516 heartbeater.cc:347] Connected to a master server at 127.18.184.190:40287
I20260808 02:15:56.254055 3516 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:56.254688 3516 heartbeater.cc:511] Master 127.18.184.190:40287 requested a full tablet report, sending...
I20260808 02:15:56.256520 3316 ts_manager.cc:194] Registered new tserver with Master: 4b89f0cc6d3b4856998cc7d081545471 (127.18.184.129:44709)
I20260808 02:15:56.258533 3316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:50827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:15:56.269712 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40287
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:56.394505 3521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:56.394796 3521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:56.394842 3521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:56.399217 3521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:56.399286 3521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:56.399374 3521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:15:56.405447 3521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:56.406682 3521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:56.407799 3521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:56.417753 3526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:56.417789 3527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:56.417837 3530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:56.418043 3521 server_base.cc:1034] running on GCE node
I20260808 02:15:56.418481 3521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:56.419252 3521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:56.420447 3521 hybrid_clock.cc:648] HybridClock initialized: now 1786155356420367 us; error 91 us; skew 500 ppm
Aug 08 02:15:56 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155356, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:56.423705 3521 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:15:56.424903 3521 webserver.cc:460] Webserver started at http://127.18.184.130:41967/ using document root <none> and password file <none>
I20260808 02:15:56.425659 3521 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:56.425755 3521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:56.426023 3521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:56.428212 3521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "321edff411a349b0860edb469cdb7128"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "a82621d6a047664c44e31da6f941fdef"
server_key_iv: "2f5dc4e913ad1af023d3f39023ef8f00"
server_key_version: "encryptionkey@0"
I20260808 02:15:56.428807 3521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "321edff411a349b0860edb469cdb7128"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "a82621d6a047664c44e31da6f941fdef"
server_key_iv: "2f5dc4e913ad1af023d3f39023ef8f00"
server_key_version: "encryptionkey@0"
I20260808 02:15:56.433056 3521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 02:15:56.435606 3538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:56.437016 3521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:15:56.437198 3521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "321edff411a349b0860edb469cdb7128"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "a82621d6a047664c44e31da6f941fdef"
server_key_iv: "2f5dc4e913ad1af023d3f39023ef8f00"
server_key_version: "encryptionkey@0"
I20260808 02:15:56.437357 3521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:56.455938 3521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:56.459268 3521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:56.459439 3521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:56.460026 3521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:56.460863 3521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:56.460912 3521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:56.460989 3521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:56.461032 3521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:56.470481 3521 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:39415
I20260808 02:15:56.470501 3662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:39415 every 8 connection(s)
I20260808 02:15:56.471474 3521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:15:56.477977 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 3521
I20260808 02:15:56.478154 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:15:56.478482 19170 external_mini_cluster.cc:1428] Setting key 820c0bfc8a6d4c666ec9378cd36bd7c5
Aug 08 02:15:56 dist-test-slave-9gcw krb5kdc[3264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155356, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:56.487675 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.473466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:53391 (local address 127.18.184.190:40287)
0808 02:15:56.473618 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:56.473622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:56.474537 (+ 915us) server_negotiation.cc:407] Connection header received
0808 02:15:56.476042 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:56.476046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:56.476116 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:56.476226 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:56.477617 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.478265 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.479578 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.479830 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.479947 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.483006 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:56.483013 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:56.483019 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:56.483058 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:56.484698 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:56.485576 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:56.485579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:56.485581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:56.485679 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:56.486227 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:56.486231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:56.486232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:56.486454 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:56.486541 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:56.487405 (+ 864us) server_negotiation.cc:299] Negotiation successful
0808 02:15:56.487539 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:15:56.489125 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.473858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:53391)
0808 02:15:56.474363 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:56.474412 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:56.475782 (+ 1370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:56.476431 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:56.476437 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:56.476842 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:56.477453 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:56.477469 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.478444 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.478446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:56.479440 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:56.479451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.480013 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.480017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:56.480215 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.480908 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:56.480926 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:56.482792 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:56.484914 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:56.484924 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:56.484947 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:56.485431 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:56.485805 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:56.485811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:56.485816 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:56.486051 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:56.486584 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:56.486594 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:56.487036 (+ 442us) client_negotiation.cc:769] Sending connection context
0808 02:15:56.487304 (+ 268us) client_negotiation.cc:240] Negotiation successful
0808 02:15:56.487851 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":118,"threads_started":1}
I20260808 02:15:56.490646 3663 heartbeater.cc:347] Connected to a master server at 127.18.184.190:40287
I20260808 02:15:56.490936 3663 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:15:56.491500 3663 heartbeater.cc:511] Master 127.18.184.190:40287 requested a full tablet report, sending...
I20260808 02:15:56.492661 3316 ts_manager.cc:194] Registered new tserver with Master: 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415)
I20260808 02:15:56.493489 3316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:53391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:15:56.513059 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40287
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:56.631568 3669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:56.631855 3669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:56.631911 3669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:56.636119 3669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:56.636189 3669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:56.636250 3669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:15:56.641955 3669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:40287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:15:56.643036 3669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:56.644039 3669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:56.654122 3675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:56.654147 3676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:56.654124 3678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:56.654400 3669 server_base.cc:1034] running on GCE node
I20260808 02:15:56.654842 3669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:56.655473 3669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:56.656698 3669 hybrid_clock.cc:648] HybridClock initialized: now 1786155356656670 us; error 45 us; skew 500 ppm
Aug 08 02:15:56 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155356, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:56.659852 3669 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:15:56.660848 3669 webserver.cc:460] Webserver started at http://127.18.184.131:43573/ using document root <none> and password file <none>
I20260808 02:15:56.661568 3669 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:56.661664 3669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:56.661907 3669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:56.664103 3669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "3d240237384a4e16a90b2846709f44d0"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "b6508d4a13f74fc3b376fc8830f78e58"
server_key_iv: "2c16932f9be1213895f4911d580b45ee"
server_key_version: "encryptionkey@0"
I20260808 02:15:56.664707 3669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "3d240237384a4e16a90b2846709f44d0"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "b6508d4a13f74fc3b376fc8830f78e58"
server_key_iv: "2c16932f9be1213895f4911d580b45ee"
server_key_version: "encryptionkey@0"
I20260808 02:15:56.668933 3669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 02:15:56.671908 3688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:56.673408 3669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:56.673571 3669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "3d240237384a4e16a90b2846709f44d0"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "b6508d4a13f74fc3b376fc8830f78e58"
server_key_iv: "2c16932f9be1213895f4911d580b45ee"
server_key_version: "encryptionkey@0"
I20260808 02:15:56.673719 3669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:56.688177 3669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:56.691745 3669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:56.691987 3669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:56.692773 3669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:56.693804 3669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:56.693887 3669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:56.693979 3669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:56.694038 3669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:56.704375 3669 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:43707
I20260808 02:15:56.704399 3817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:43707 every 8 connection(s)
I20260808 02:15:56.705514 3669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:15:56.711284 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 3669
I20260808 02:15:56.711453 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:15:56.711781 19170 external_mini_cluster.cc:1428] Setting key 9c7aa76039dd65e9995cd6a21adda472
Aug 08 02:15:56 dist-test-slave-9gcw krb5kdc[3264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155356, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:15:56.720636 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.707536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:33913 (local address 127.18.184.190:40287)
0808 02:15:56.707705 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:56.707708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:56.708643 (+ 935us) server_negotiation.cc:407] Connection header received
0808 02:15:56.710147 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:56.710150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:56.710214 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:56.710326 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:56.711641 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.712319 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.714007 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.714346 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.714436 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.716994 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:56.716997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:56.716999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:56.717025 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:56.718456 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:56.718999 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:56.719002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:56.719003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:56.719093 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:56.719433 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:56.719436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:56.719437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:56.719617 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:56.719715 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:56.720398 (+ 683us) server_negotiation.cc:299] Negotiation successful
0808 02:15:56.720508 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:15:56.721588 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.707950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:33913)
0808 02:15:56.708476 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:56.708524 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:56.709921 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:56.710514 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:56.710520 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:56.710918 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:56.711467 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:56.711488 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.712463 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.712466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:56.713818 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:56.713829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.714493 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.714496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:56.714642 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.715311 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:56.715336 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:56.716834 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:56.718598 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:56.718603 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:56.718615 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:56.718900 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:56.719204 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:56.719206 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:56.719207 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:56.719347 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:56.719717 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:56.719722 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:56.720164 (+ 442us) client_negotiation.cc:769] Sending connection context
0808 02:15:56.720333 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 02:15:56.720597 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":123,"threads_started":1}
I20260808 02:15:56.722868 3819 heartbeater.cc:347] Connected to a master server at 127.18.184.190:40287
I20260808 02:15:56.723141 3819 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:56.723675 3819 heartbeater.cc:511] Master 127.18.184.190:40287 requested a full tablet report, sending...
I20260808 02:15:56.724895 3316 ts_manager.cc:194] Registered new tserver with Master: 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707)
I20260808 02:15:56.725541 3316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:33913
I20260808 02:15:56.726262 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:15:56.736049 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.728335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54838 (local address 127.18.184.190:40287)
0808 02:15:56.728468 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:56.728472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:56.728679 (+ 207us) server_negotiation.cc:407] Connection header received
0808 02:15:56.728788 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:56.728790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:56.728831 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:56.728893 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:56.729670 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.730184 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.731058 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.731313 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.731481 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.733095 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:56.733098 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:56.733100 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:56.733125 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:56.734481 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:56.734874 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:56.734876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:56.734877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:56.734952 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:56.735175 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:56.735177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:56.735178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:56.735486 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:56.735626 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:56.735817 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 02:15:56.735922 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:15:56.739179 3315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:15:56.742532 3315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:15:56.758373 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.752413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:44709 (local address 127.0.0.1:58660)
0808 02:15:56.753323 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:56.753337 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:56.753429 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:56.753986 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:56.753988 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:56.754005 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:56.754105 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:56.754110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.755344 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.755349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:56.756650 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:56.756660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.757751 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.757756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:56.757872 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.757918 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 02:15:56.758048 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 02:15:56.758185 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"mutex_wait_us":198,"thread_start_us":141,"threads_started":1}
I20260808 02:15:56.758701 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.751712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:39415 (local address 127.0.0.1:49176)
0808 02:15:56.752729 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:56.752984 (+ 255us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:56.753124 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:56.753789 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:56.753794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:56.753816 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:56.753974 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:56.753981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.755501 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.755504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:56.757078 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:56.757084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.758288 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.758291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:56.758370 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.758402 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:15:56.758477 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 02:15:56.758556 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":197,"thread_start_us":117,"threads_started":1}
I20260808 02:15:56.758948 3830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.751633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:43707 (local address 127.0.0.1:48930)
0808 02:15:56.752952 (+ 1319us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:56.753270 (+ 318us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:56.753364 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:56.754105 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:56.754108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:56.754134 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:56.754255 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:56.754264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.755650 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.755653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:56.757073 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:56.757084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.758462 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.758465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:56.758567 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.758599 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:15:56.758678 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 02:15:56.758762 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":151,"thread_start_us":107,"threads_started":1}
I20260808 02:15:56.759117 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.752659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58660 (local address 127.18.184.129:44709)
0808 02:15:56.753061 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:56.753076 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:56.753473 (+ 397us) server_negotiation.cc:407] Connection header received
0808 02:15:56.753600 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:56.753604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:56.753794 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:56.753946 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:56.754229 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.754997 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.756805 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.757606 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.757749 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.757893 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:56.758101 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 02:15:56.758222 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":119,"threads_started":1}
I20260808 02:15:56.759155 3833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.751999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49176 (local address 127.18.184.130:39415)
0808 02:15:56.752432 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:56.752447 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:56.753137 (+ 690us) server_negotiation.cc:407] Connection header received
0808 02:15:56.753357 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:56.753366 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:56.753627 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:56.753797 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:56.754110 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.755330 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.757272 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.757994 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.758110 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.758225 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:56.758877 (+ 652us) server_negotiation.cc:299] Negotiation successful
0808 02:15:56.758993 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":115,"threads_started":1}
I20260808 02:15:56.759523 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:56.751937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48930 (local address 127.18.184.131:43707)
0808 02:15:56.753106 (+ 1169us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:56.753483 (+ 377us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:56.753507 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:15:56.753599 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:56.753604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:56.753794 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:56.753946 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:56.754397 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.755511 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.757256 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:56.758107 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:56.758241 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:56.758385 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:56.759256 (+ 871us) server_negotiation.cc:299] Negotiation successful
0808 02:15:56.759371 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1059,"thread_start_us":105,"threads_started":1}
I20260808 02:15:56.762203 3742 tablet_service.cc:1467] Processing CreateTablet for tablet ab99dd0e28bb4eceaeaef2f3dde630b0 (DEFAULT_TABLE table=test-table [id=4cef6fe1558b4427ba40328b09371495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:56.762202 3447 tablet_service.cc:1467] Processing CreateTablet for tablet ab99dd0e28bb4eceaeaef2f3dde630b0 (DEFAULT_TABLE table=test-table [id=4cef6fe1558b4427ba40328b09371495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:56.762205 3590 tablet_service.cc:1467] Processing CreateTablet for tablet ab99dd0e28bb4eceaeaef2f3dde630b0 (DEFAULT_TABLE table=test-table [id=4cef6fe1558b4427ba40328b09371495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:56.763334 3590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab99dd0e28bb4eceaeaef2f3dde630b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:56.763336 3447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab99dd0e28bb4eceaeaef2f3dde630b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:56.763348 3742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab99dd0e28bb4eceaeaef2f3dde630b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:56.770040 3838 tablet_bootstrap.cc:492] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0: Bootstrap starting.
I20260808 02:15:56.771327 3839 tablet_bootstrap.cc:492] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Bootstrap starting.
I20260808 02:15:56.772462 3838 tablet_bootstrap.cc:654] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:56.772989 3840 tablet_bootstrap.cc:492] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128: Bootstrap starting.
I20260808 02:15:56.773159 3839 tablet_bootstrap.cc:654] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:56.774031 3839 log.cc:824] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:56.774032 3838 log.cc:824] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:56.775641 3840 tablet_bootstrap.cc:654] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:56.776213 3838 tablet_bootstrap.cc:492] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0: No bootstrap required, opened a new log
I20260808 02:15:56.776213 3839 tablet_bootstrap.cc:492] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: No bootstrap required, opened a new log
I20260808 02:15:56.776417 3839 ts_tablet_manager.cc:1397] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 02:15:56.776417 3838 ts_tablet_manager.cc:1397] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 02:15:56.776443 3840 log.cc:824] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:56.778394 3840 tablet_bootstrap.cc:492] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128: No bootstrap required, opened a new log
I20260808 02:15:56.778596 3840 ts_tablet_manager.cc:1397] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:15:56.780730 3838 raft_consensus.cc:348] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:15:56.781037 3838 raft_consensus.cc:374] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:56.781085 3838 raft_consensus.cc:729] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d240237384a4e16a90b2846709f44d0, State: Initialized, Role: FOLLOWER
I20260808 02:15:56.780992 3839 raft_consensus.cc:348] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:15:56.781273 3839 raft_consensus.cc:374] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:56.781348 3839 raft_consensus.cc:729] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b89f0cc6d3b4856998cc7d081545471, State: Initialized, Role: FOLLOWER
I20260808 02:15:56.781682 3838 consensus_queue.cc:260] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:15:56.781718 3839 consensus_queue.cc:260] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:15:56.781978 3840 raft_consensus.cc:348] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:15:56.782258 3840 raft_consensus.cc:374] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:56.782336 3840 raft_consensus.cc:729] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 321edff411a349b0860edb469cdb7128, State: Initialized, Role: FOLLOWER
I20260808 02:15:56.782781 3838 ts_tablet_manager.cc:1428] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:15:56.782781 3839 ts_tablet_manager.cc:1428] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:15:56.782822 3840 consensus_queue.cc:260] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:15:56.782912 3516 heartbeater.cc:503] Master 127.18.184.190:40287 was elected leader, sending a full tablet report...
I20260808 02:15:56.783511 3663 heartbeater.cc:503] Master 127.18.184.190:40287 was elected leader, sending a full tablet report...
I20260808 02:15:56.783833 3840 ts_tablet_manager.cc:1428] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 02:15:56.784157 3819 heartbeater.cc:503] Master 127.18.184.190:40287 was elected leader, sending a full tablet report...
W20260808 02:15:56.956585 3820 tablet.cc:2369] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:15:56.972811 3664 tablet.cc:2369] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:15:56.985962 3517 tablet.cc:2369] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:15:57.084934 3847 raft_consensus.cc:482] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:57.085466 3847 raft_consensus.cc:504] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:15:57.086782 3847 leader_election.cc:288] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707), 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415)
I20260808 02:15:57.093225 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:57.087387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:33999 (local address 127.18.184.131:43707)
0808 02:15:57.087542 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:57.087546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:57.087876 (+ 330us) server_negotiation.cc:407] Connection header received
0808 02:15:57.088065 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:57.088069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:57.088128 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:57.088259 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:57.088688 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:57.089610 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:57.091732 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:57.092407 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:57.092545 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:57.092600 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:57.092943 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 02:15:57.093059 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:15:57.093271 3833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:57.087278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:34301 (local address 127.18.184.130:39415)
0808 02:15:57.087446 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:57.087451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:57.087882 (+ 431us) server_negotiation.cc:407] Connection header received
0808 02:15:57.088066 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:57.088069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:57.088130 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:57.088228 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:57.088690 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:57.089608 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:57.091778 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:57.092407 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:57.092545 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:57.092600 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:57.093050 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 02:15:57.093162 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 02:15:57.093266 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:57.087184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:39415 (local address 127.18.184.129:34301)
0808 02:15:57.087746 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:57.087762 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:57.087930 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:57.088263 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:57.088266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:57.088317 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:57.088552 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:57.088558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:57.089794 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:57.089798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:57.091533 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:57.091547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:57.092573 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:57.092579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:57.092885 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:57.092927 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 02:15:57.093001 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:15:57.093056 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":315,"spinlock_wait_cycles":23936,"thread_start_us":157,"threads_started":1}
I20260808 02:15:57.093274 3850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:57.087215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:43707 (local address 127.18.184.129:33999)
0808 02:15:57.087738 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:57.087758 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:57.087930 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:57.088263 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:57.088266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:57.088317 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:57.088536 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:57.088544 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:57.089794 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:57.089798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:57.091533 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:57.091547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:57.092573 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:57.092579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:57.092752 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:57.092791 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 02:15:57.092910 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 02:15:57.093010 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":249,"spinlock_wait_cycles":42880,"thread_start_us":119,"threads_started":1}
I20260808 02:15:57.094185 3614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab99dd0e28bb4eceaeaef2f3dde630b0" candidate_uuid: "4b89f0cc6d3b4856998cc7d081545471" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "321edff411a349b0860edb469cdb7128" is_pre_election: true
I20260808 02:15:57.094185 3765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab99dd0e28bb4eceaeaef2f3dde630b0" candidate_uuid: "4b89f0cc6d3b4856998cc7d081545471" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d240237384a4e16a90b2846709f44d0" is_pre_election: true
I20260808 02:15:57.094539 3765 raft_consensus.cc:2454] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b89f0cc6d3b4856998cc7d081545471 in term 0.
I20260808 02:15:57.094539 3614 raft_consensus.cc:2454] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b89f0cc6d3b4856998cc7d081545471 in term 0.
I20260808 02:15:57.095113 3400 leader_election.cc:302] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 321edff411a349b0860edb469cdb7128, 3d240237384a4e16a90b2846709f44d0, 4b89f0cc6d3b4856998cc7d081545471; no voters:
I20260808 02:15:57.095403 3847 raft_consensus.cc:2781] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:15:57.095492 3847 raft_consensus.cc:482] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:15:57.095535 3847 raft_consensus.cc:3037] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:57.096629 3847 raft_consensus.cc:504] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:15:57.096969 3847 leader_election.cc:288] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 1 election: Requested vote from peers 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707), 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415)
I20260808 02:15:57.097363 3765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab99dd0e28bb4eceaeaef2f3dde630b0" candidate_uuid: "4b89f0cc6d3b4856998cc7d081545471" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d240237384a4e16a90b2846709f44d0"
I20260808 02:15:57.097380 3614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab99dd0e28bb4eceaeaef2f3dde630b0" candidate_uuid: "4b89f0cc6d3b4856998cc7d081545471" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "321edff411a349b0860edb469cdb7128"
I20260808 02:15:57.097506 3765 raft_consensus.cc:3037] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:57.097514 3614 raft_consensus.cc:3037] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:57.098661 3614 raft_consensus.cc:2454] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b89f0cc6d3b4856998cc7d081545471 in term 1.
I20260808 02:15:57.098749 3765 raft_consensus.cc:2454] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b89f0cc6d3b4856998cc7d081545471 in term 1.
I20260808 02:15:57.099030 3400 leader_election.cc:302] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 321edff411a349b0860edb469cdb7128, 4b89f0cc6d3b4856998cc7d081545471; no voters:
I20260808 02:15:57.099263 3847 raft_consensus.cc:2781] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:57.099541 3847 raft_consensus.cc:686] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 LEADER]: Becoming Leader. State: Replica: 4b89f0cc6d3b4856998cc7d081545471, State: Running, Role: LEADER
I20260808 02:15:57.099879 3847 consensus_queue.cc:237] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:15:57.103252 3316 catalog_manager.cc:5495] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b89f0cc6d3b4856998cc7d081545471 (127.18.184.129). New cstate: current_term: 1 leader_uuid: "4b89f0cc6d3b4856998cc7d081545471" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } health_report { overall_health: HEALTHY } } }
I20260808 02:15:57.124442 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:57.120255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.18.184.129:44709)
0808 02:15:57.120409 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:57.120415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:57.120431 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:15:57.120573 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:57.120579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:57.120648 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:57.120838 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:15:57.121163 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:57.121754 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:57.122750 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:57.123006 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:57.123130 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:57.123511 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:15:57.123986 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:15:57.124141 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:57.124189 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 02:15:57.124261 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:15:57.130915 3614 raft_consensus.cc:1261] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128 [term 1 FOLLOWER]: Refusing update from remote peer 4b89f0cc6d3b4856998cc7d081545471: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:57.130915 3765 raft_consensus.cc:1261] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0 [term 1 FOLLOWER]: Refusing update from remote peer 4b89f0cc6d3b4856998cc7d081545471: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:57.131724 3847 consensus_queue.cc:1035] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:57.131937 3851 consensus_queue.cc:1035] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [LEADER]: Connected to new peer: Peer: permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:57.141593 3856 mvcc.cc:205] Tried to move back new op lower bound from 7316092342800146432 to 7316092342683734016. Current Snapshot: MvccSnapshot[applied={T|T < 7316092342800146432}]
I20260808 02:15:57.142428 3855 mvcc.cc:205] Tried to move back new op lower bound from 7316092342800146432 to 7316092342683734016. Current Snapshot: MvccSnapshot[applied={T|T < 7316092342800146432}]
I20260808 02:15:57.148656 3316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:57.148849 3316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:57.151525 3316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Sending DeleteTablet for 3 replicas of tablet ab99dd0e28bb4eceaeaef2f3dde630b0
I20260808 02:15:57.152364 3742 tablet_service.cc:1514] Processing DeleteTablet for tablet ab99dd0e28bb4eceaeaef2f3dde630b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:57 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:48930
I20260808 02:15:57.152423 3590 tablet_service.cc:1514] Processing DeleteTablet for tablet ab99dd0e28bb4eceaeaef2f3dde630b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:57 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:49176
I20260808 02:15:57.152420 3447 tablet_service.cc:1514] Processing DeleteTablet for tablet ab99dd0e28bb4eceaeaef2f3dde630b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:57 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:58660
I20260808 02:15:57.152923 3865 tablet_replica.cc:330] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0: stopping tablet replica
I20260808 02:15:57.153121 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 3381
I20260808 02:15:57.153105 3866 tablet_replica.cc:330] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128: stopping tablet replica
I20260808 02:15:57.156271 3866 raft_consensus.cc:2229] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:57.156744 3866 raft_consensus.cc:2258] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:57.157348 3865 raft_consensus.cc:2229] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:57.157655 3865 raft_consensus.cc:2258] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:57.158434 3866 ts_tablet_manager.cc:1905] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:57.159073 3865 ts_tablet_manager.cc:1905] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:57.160931 3866 ts_tablet_manager.cc:1918] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:57.161047 3866 log.cc:1196] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/ab99dd0e28bb4eceaeaef2f3dde630b0
I20260808 02:15:57.161448 3866 ts_tablet_manager.cc:1939] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 321edff411a349b0860edb469cdb7128: Deleting consensus metadata
I20260808 02:15:57.162732 3865 ts_tablet_manager.cc:1918] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:57.162837 3865 log.cc:1196] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/ab99dd0e28bb4eceaeaef2f3dde630b0
I20260808 02:15:57.162757 3299 catalog_manager.cc:4841] TS 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415): tablet ab99dd0e28bb4eceaeaef2f3dde630b0 (table test-table [id=4cef6fe1558b4427ba40328b09371495]) successfully deleted
I20260808 02:15:57.163179 3865 ts_tablet_manager.cc:1939] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 3d240237384a4e16a90b2846709f44d0: Deleting consensus metadata
I20260808 02:15:57.164767 3301 catalog_manager.cc:4841] TS 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707): tablet ab99dd0e28bb4eceaeaef2f3dde630b0 (table test-table [id=4cef6fe1558b4427ba40328b09371495]) successfully deleted
I20260808 02:15:57.164855 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 3521
W20260808 02:15:57.164779 3299 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:44709 (error 108)
I20260808 02:15:57.168572 3830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:57.168286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:44709 (local address 127.0.0.1:58688)
0808 02:15:57.168426 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:57.168492 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 02:15:57.168846 3299 catalog_manager.cc:4584] TS 4b89f0cc6d3b4856998cc7d081545471 (127.18.184.129:44709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab99dd0e28bb4eceaeaef2f3dde630b0: Network error: Client connection negotiation failed: client connection to 127.18.184.129:44709: connect: Connection refused (error 111)
I20260808 02:15:57.174194 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 3669
I20260808 02:15:57.181123 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 3278
I20260808 02:16:17.190985 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:40287
--webserver_interface=127.18.184.190
--webserver_port=34609
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:40287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:17.313081 3869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:17.313412 3869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:17.313453 3869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:17.317595 3869 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 02:16:17.317663 3869 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 02:16:17.317680 3869 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 02:16:17.317694 3869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:17.317714 3869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:17.317732 3869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:17.317749 3869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:17.317763 3869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:17.323374 3869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:40287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:40287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=34609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:17.324450 3869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:17.325548 3869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:17.340993 3876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:17.340998 3877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:17.341064 3869 server_base.cc:1034] running on GCE node
W20260808 02:16:17.340996 3879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:17.342056 3869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:17.343365 3869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:17.344595 3869 hybrid_clock.cc:648] HybridClock initialized: now 1786155377344553 us; error 60 us; skew 500 ppm
Aug 08 02:16:17 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:17.352049 3869 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:16:17.353260 3869 webserver.cc:460] Webserver started at http://127.18.184.190:34609/ using document root <none> and password file <none>
I20260808 02:16:17.353943 3869 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:17.354072 3869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:17.358656 3869 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:16:17.361948 3886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:17.363391 3869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:16:17.363562 3869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "a4228483587740a1a8333148c6b43e66"
format_stamp: "Formatted at 2026-08-08 02:15:55 on dist-test-slave-9gcw"
server_key: "7759432198d95601bc323e16a15151a5"
server_key_iv: "1f6f745e5d893c1fd826ebe4e00b2f76"
server_key_version: "encryptionkey@0"
I20260808 02:16:17.364352 3869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:17.381584 3869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:17.385408 3869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:17.385771 3869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:17.394243 3869 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:40287
I20260808 02:16:17.394583 3944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:40287 every 8 connection(s)
I20260808 02:16:17.395558 3869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:17.400444 3945 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:16:17.400561 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 3869
I20260808 02:16:17.401026 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:44709
--local_ip_for_outbound_sockets=127.18.184.129
--tserver_master_addrs=127.18.184.190:40287
--webserver_port=39235
--webserver_interface=127.18.184.129
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:16:17.402400 3945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Bootstrap starting.
I20260808 02:16:17.421684 3945 log.cc:824] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:17.431802 3945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:16:17.432294 3945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Bootstrap complete.
I20260808 02:16:17.436735 3945 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } }
I20260808 02:16:17.437227 3945 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4228483587740a1a8333148c6b43e66, State: Initialized, Role: FOLLOWER
I20260808 02:16:17.437755 3945 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } }
I20260808 02:16:17.437879 3945 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:17.437994 3945 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:17.438134 3945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:16:17.440312 3945 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } }
I20260808 02:16:17.440691 3945 leader_election.cc:302] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4228483587740a1a8333148c6b43e66; no voters:
I20260808 02:16:17.441071 3945 leader_election.cc:288] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:16:17.441201 3949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:16:17.441428 3949 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 2 LEADER]: Becoming Leader. State: Replica: a4228483587740a1a8333148c6b43e66, State: Running, Role: LEADER
I20260808 02:16:17.441830 3949 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } }
I20260808 02:16:17.442445 3945 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:17.443465 3951 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4228483587740a1a8333148c6b43e66. Latest consensus state: current_term: 2 leader_uuid: "a4228483587740a1a8333148c6b43e66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } } }
I20260808 02:16:17.443615 3951 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:17.443516 3950 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a4228483587740a1a8333148c6b43e66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } } }
I20260808 02:16:17.443674 3950 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:17.448244 3965 catalog_manager.cc:1259] Loaded cluster ID: ad53c1bc050d4f99b6dde1ab60fee4bd
I20260808 02:16:17.448324 3965 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: loading cluster ID for follower catalog manager: success
I20260808 02:16:17.450937 3965 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: acquiring CA information for follower catalog manager: success
I20260808 02:16:17.452652 3965 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:16:17.452808 3958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:17.455518 3958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:17.456089 3958 catalog_manager.cc:1259] Loaded cluster ID: ad53c1bc050d4f99b6dde1ab60fee4bd
I20260808 02:16:17.456203 3958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:17.457016 3958 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:17.457799 3958 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Loaded TSK: 0
I20260808 02:16:17.473296 3958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Generated new TSK 1
I20260808 02:16:17.473623 3958 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:16:17.525876 3947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:17.526216 3947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:17.526309 3947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:17.530707 3947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:17.530838 3947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:17.530941 3947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:16:17.536773 3947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:44709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=39235
--tserver_master_addrs=127.18.184.190:40287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:17.538044 3947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:17.539172 3947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:17.550407 3975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:17.550454 3974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:17.550465 3977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:17.550765 3947 server_base.cc:1034] running on GCE node
I20260808 02:16:17.551373 3947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:17.552274 3947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:17.553483 3947 hybrid_clock.cc:648] HybridClock initialized: now 1786155377553456 us; error 37 us; skew 500 ppm
Aug 08 02:16:17 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:17.557367 3947 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:16:17.558609 3947 webserver.cc:460] Webserver started at http://127.18.184.129:39235/ using document root <none> and password file <none>
I20260808 02:16:17.559470 3947 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:17.559594 3947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:17.565227 3947 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:16:17.568411 3984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:17.570576 3947 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260808 02:16:17.570765 3947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "4b89f0cc6d3b4856998cc7d081545471"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "baba4c5d31da52385d14ddb4ec59ea6c"
server_key_iv: "96386123645954c31047f5c0386a110c"
server_key_version: "encryptionkey@0"
I20260808 02:16:17.571432 3947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:17.596112 3947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:17.599881 3947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:17.600122 3947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:17.600850 3947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:17.602121 3991 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:16:17.608037 3947 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:16:17.608263 3947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 02:16:17.608364 3947 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:16:17.610226 3947 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:16:17.610306 3947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 02:16:17.610620 3991 tablet_bootstrap.cc:492] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Bootstrap starting.
I20260808 02:16:17.621915 3947 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:44709
I20260808 02:16:17.622017 4109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:44709 every 8 connection(s)
I20260808 02:16:17.623013 3947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:16:17.630498 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 3947
I20260808 02:16:17.631106 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:39415
--local_ip_for_outbound_sockets=127.18.184.130
--tserver_master_addrs=127.18.184.190:40287
--webserver_port=41967
--webserver_interface=127.18.184.130
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:16:17.632714 3991 log.cc:824] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:16:17 dist-test-slave-9gcw krb5kdc[3264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:16:17.642473 3991 tablet_bootstrap.cc:492] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:16:17.642985 3991 tablet_bootstrap.cc:492] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Bootstrap complete.
I20260808 02:16:17.644785 3991 ts_tablet_manager.cc:1397] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Time spent bootstrapping tablet: real 0.034s user 0.024s sys 0.009s
I20260808 02:16:17.645315 4113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:17.625376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:57959 (local address 127.18.184.190:40287)
0808 02:16:17.626059 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:17.626081 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:17.626136 (+ 55us) server_negotiation.cc:407] Connection header received
0808 02:16:17.627379 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:17.627404 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:17.627862 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:17.628384 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:17.629686 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:17.631509 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:17.633485 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:17.634334 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:17.634553 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:17.637942 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:17.637950 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:17.637965 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:17.638004 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:17.640443 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:17.641352 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:17.641359 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:17.641364 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:17.641538 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:17.642158 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:17.642162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:17.642164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:17.642696 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:17.642852 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:17.643735 (+ 883us) server_negotiation.cc:299] Negotiation successful
0808 02:16:17.644077 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":226,"threads_started":1}
I20260808 02:16:17.645339 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:17.625338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:57959)
0808 02:16:17.625859 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:17.625917 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:17.626815 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:17.628431 (+ 1616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:17.628438 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:17.628837 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:17.629467 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:17.629487 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:17.631706 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:17.631712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:17.633299 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:17.633316 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:17.634540 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:17.634544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:17.634745 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:17.635510 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:17.635531 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:17.637683 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:17.640630 (+ 2947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:17.640640 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:17.640657 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:17.641182 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:17.641760 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:17.641765 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:17.641769 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:17.642000 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:17.642849 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:17.642860 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:17.643301 (+ 441us) client_negotiation.cc:769] Sending connection context
0808 02:16:17.643566 (+ 265us) client_negotiation.cc:240] Negotiation successful
0808 02:16:17.644009 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":147,"threads_started":1}
I20260808 02:16:17.647876 3991 raft_consensus.cc:348] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:17.648489 3991 raft_consensus.cc:729] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b89f0cc6d3b4856998cc7d081545471, State: Initialized, Role: FOLLOWER
I20260808 02:16:17.648716 4110 heartbeater.cc:347] Connected to a master server at 127.18.184.190:40287
I20260808 02:16:17.648931 4110 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:17.649000 3991 consensus_queue.cc:260] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:17.649621 4110 heartbeater.cc:511] Master 127.18.184.190:40287 requested a full tablet report, sending...
I20260808 02:16:17.650090 3991 ts_tablet_manager.cc:1428] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 02:16:17.652359 3906 ts_manager.cc:194] Registered new tserver with Master: 4b89f0cc6d3b4856998cc7d081545471 (127.18.184.129:44709)
I20260808 02:16:17.656029 3906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:57959
I20260808 02:16:17.658758 4110 heartbeater.cc:503] Master 127.18.184.190:40287 was elected leader, sending a full tablet report...
Aug 08 02:16:17 dist-test-slave-9gcw krb5kdc[3264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.129@KRBTEST.COM
I20260808 02:16:17.668254 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:17.655461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:44709 (local address 127.0.0.1:55740)
0808 02:16:17.655905 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:17.655955 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:17.656107 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:17.657032 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:17.657038 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:17.657558 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:17.657698 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:17.657708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:17.659100 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:17.659104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:17.660011 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:17.660022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:17.660968 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:17.660971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:17.661056 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:17.661768 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:17.661781 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:17.663519 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:17.666203 (+ 2684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:17.666215 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:17.666219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:17.666539 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:17.666943 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:17.666946 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:17.666947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:17.667038 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:17.667643 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:17.667649 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:17.667771 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 02:16:17.667902 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 02:16:17.668089 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":133,"threads_started":1}
I20260808 02:16:17.668382 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:17.655582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55740 (local address 127.18.184.129:44709)
0808 02:16:17.656360 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:17.656371 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:17.656395 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:16:17.656468 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:17.656473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:17.656706 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:17.657020 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:17.657852 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:17.658963 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:17.660150 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:17.660840 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:17.660950 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:17.663713 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:17.663727 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:17.663742 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:17.663773 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:17.666047 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:17.666688 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:17.666694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:17.666707 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:17.666826 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:17.667140 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:17.667143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:17.667145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:17.667518 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:17.667662 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:17.668012 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 02:16:17.668213 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":110,"threads_started":1}
I20260808 02:16:17.669667 4035 tablet_service.cc:1514] Processing DeleteTablet for tablet ab99dd0e28bb4eceaeaef2f3dde630b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:57 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:55740
I20260808 02:16:17.670641 4119 tablet_replica.cc:330] stopping tablet replica
I20260808 02:16:17.670990 4119 raft_consensus.cc:2229] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:17.671260 4119 raft_consensus.cc:2258] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:17.672473 4119 ts_tablet_manager.cc:1905] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:17.677577 4119 ts_tablet_manager.cc:1918] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:17.677702 4119 log.cc:1196] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal/wals/ab99dd0e28bb4eceaeaef2f3dde630b0
I20260808 02:16:17.678022 4119 ts_tablet_manager.cc:1939] T ab99dd0e28bb4eceaeaef2f3dde630b0 P 4b89f0cc6d3b4856998cc7d081545471: Deleting consensus metadata
I20260808 02:16:17.680176 3888 catalog_manager.cc:4841] TS 4b89f0cc6d3b4856998cc7d081545471 (127.18.184.129:44709): tablet ab99dd0e28bb4eceaeaef2f3dde630b0 (table test-table [id=4cef6fe1558b4427ba40328b09371495]) successfully deleted
W20260808 02:16:17.763921 4114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:17.764237 4114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:17.764283 4114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:17.768469 4114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:17.768544 4114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:17.768666 4114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:16:17.774300 4114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:39415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=41967
--tserver_master_addrs=127.18.184.190:40287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:17.775592 4114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:17.776738 4114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:17.787685 4125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:17.787742 4127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:17.787742 4124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:17.787953 4114 server_base.cc:1034] running on GCE node
I20260808 02:16:17.788362 4114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:17.789027 4114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:17.790197 4114 hybrid_clock.cc:648] HybridClock initialized: now 1786155377790169 us; error 45 us; skew 500 ppm
Aug 08 02:16:17 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:17.793044 4114 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:16:17.794060 4114 webserver.cc:460] Webserver started at http://127.18.184.130:41967/ using document root <none> and password file <none>
I20260808 02:16:17.794588 4114 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:17.794662 4114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:17.798609 4114 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:16:17.800872 4135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:17.802198 4114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:16:17.802343 4114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "321edff411a349b0860edb469cdb7128"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "a82621d6a047664c44e31da6f941fdef"
server_key_iv: "2f5dc4e913ad1af023d3f39023ef8f00"
server_key_version: "encryptionkey@0"
I20260808 02:16:17.802932 4114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:17.840970 4114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:17.844604 4114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:17.844828 4114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:17.845544 4114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:17.846431 4114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:17.846479 4114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:17.846553 4114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:17.846589 4114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:17.856415 4114 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:39415
I20260808 02:16:17.856444 4262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:39415 every 8 connection(s)
I20260808 02:16:17.857532 4114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:16:17.861047 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 4114
I20260808 02:16:17.861579 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:43707
--local_ip_for_outbound_sockets=127.18.184.131
--tserver_master_addrs=127.18.184.190:40287
--webserver_port=43573
--webserver_interface=127.18.184.131
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:16:17 dist-test-slave-9gcw krb5kdc[3264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:16:17.873099 4113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:17.859590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:33689 (local address 127.18.184.190:40287)
0808 02:16:17.859767 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:17.859770 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:17.860612 (+ 842us) server_negotiation.cc:407] Connection header received
0808 02:16:17.862165 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:17.862172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:17.862246 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:17.862350 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:17.863558 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:17.864102 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:17.865415 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:17.865725 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:17.865859 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:17.868861 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:17.868866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:17.868868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:17.868903 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:17.870753 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:17.871354 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:17.871358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:17.871360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:17.871461 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:17.871864 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:17.871868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:17.871870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:17.872077 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:17.872194 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:17.872745 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 02:16:17.872897 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:16:17.873993 4265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:17.859969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:33689)
0808 02:16:17.860450 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:17.860493 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:17.861949 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:17.862508 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:17.862514 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:17.862877 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:17.863396 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:17.863410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:17.864250 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:17.864258 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:17.865294 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:17.865309 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:17.865907 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:17.865911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:17.866092 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:17.866744 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:17.866761 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:17.868563 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:17.870898 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:17.870904 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:17.870922 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:17.871234 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:17.871563 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:17.871566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:17.871568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:17.871736 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:17.872214 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:17.872221 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:17.872493 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 02:16:17.872696 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 02:16:17.873056 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":113,"threads_started":1}
I20260808 02:16:17.875636 4263 heartbeater.cc:347] Connected to a master server at 127.18.184.190:40287
I20260808 02:16:17.875905 4263 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:17.876570 4263 heartbeater.cc:511] Master 127.18.184.190:40287 requested a full tablet report, sending...
I20260808 02:16:17.877877 3906 ts_manager.cc:194] Registered new tserver with Master: 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415)
I20260808 02:16:17.878552 3906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:33689
W20260808 02:16:17.982687 4266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:17.983033 4266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:17.983080 4266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:17.987444 4266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:17.987522 4266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:17.987625 4266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:16:17.993408 4266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:43707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=43573
--tserver_master_addrs=127.18.184.190:40287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:17.994643 4266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:17.995736 4266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:18.006053 4273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:18.006207 4266 server_base.cc:1034] running on GCE node
W20260808 02:16:18.006053 4275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:18.006444 4272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:18.006891 4266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:18.007659 4266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:18.008885 4266 hybrid_clock.cc:648] HybridClock initialized: now 1786155378008845 us; error 62 us; skew 500 ppm
Aug 08 02:16:18 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155378, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:18.011852 4266 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:16:18.012786 4266 webserver.cc:460] Webserver started at http://127.18.184.131:43573/ using document root <none> and password file <none>
I20260808 02:16:18.013404 4266 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:18.013484 4266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:18.017530 4266 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:16:18.019794 4282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:18.021286 4266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:18.021394 4266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "3d240237384a4e16a90b2846709f44d0"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "b6508d4a13f74fc3b376fc8830f78e58"
server_key_iv: "2c16932f9be1213895f4911d580b45ee"
server_key_version: "encryptionkey@0"
I20260808 02:16:18.021961 4266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:18.053576 4266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:18.057225 4266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:18.057456 4266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:18.058154 4266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:18.059032 4266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:18.059109 4266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:18.059180 4266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:18.059224 4266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:18.068863 4266 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:43707
I20260808 02:16:18.068912 4402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:43707 every 8 connection(s)
I20260808 02:16:18.070106 4266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:16:18.080251 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 4266
Aug 08 02:16:18 dist-test-slave-9gcw krb5kdc[3264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155378, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:16:18.085824 4113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.072075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:57535 (local address 127.18.184.190:40287)
0808 02:16:18.072227 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:18.072236 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:18.073113 (+ 877us) server_negotiation.cc:407] Connection header received
0808 02:16:18.074574 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:18.074578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:18.074644 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:18.074728 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:18.076116 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.076650 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.078059 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.078305 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.078437 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.081312 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:18.081316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:18.081319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:18.081351 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:18.083273 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:18.083925 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:18.083929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:18.083932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:18.084027 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:18.084376 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:18.084379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:18.084381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:18.084598 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:18.084711 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:18.085368 (+ 657us) server_negotiation.cc:299] Negotiation successful
0808 02:16:18.085508 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:16:18.086591 4405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.072435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:57535)
0808 02:16:18.072950 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:18.072998 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:18.074336 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:18.074962 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:18.074971 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:18.075402 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:18.075919 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:18.075934 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.076797 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.076801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:18.077914 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:18.077924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.078484 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.078489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:18.078673 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.079256 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:18.079273 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:18.081059 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:18.083385 (+ 2326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:18.083391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:18.083409 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:18.083786 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:18.084120 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:18.084122 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:18.084123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:18.084266 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:18.084703 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:18.084708 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:18.085030 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 02:16:18.085256 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 02:16:18.085599 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":109,"threads_started":1}
I20260808 02:16:18.087957 4403 heartbeater.cc:347] Connected to a master server at 127.18.184.190:40287
I20260808 02:16:18.088248 4403 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:18.088852 4403 heartbeater.cc:511] Master 127.18.184.190:40287 requested a full tablet report, sending...
I20260808 02:16:18.090145 3906 ts_manager.cc:194] Registered new tserver with Master: 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707)
I20260808 02:16:18.090771 4407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.081028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36090 (local address 127.18.184.190:40287)
0808 02:16:18.081370 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:18.081374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:18.081553 (+ 179us) server_negotiation.cc:407] Connection header received
0808 02:16:18.081720 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:18.081726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:18.081777 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:18.081882 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:18.082906 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.083651 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.084879 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.085242 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.085351 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.087161 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:18.087164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:18.087166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:18.087188 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:18.088859 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:18.089381 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:18.089384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:18.089385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:18.089486 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:18.089794 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:18.089797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:18.089798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:18.090037 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:18.090244 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:18.090481 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 02:16:18.090606 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":112,"threads_started":1}
I20260808 02:16:18.090904 3906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:57535
I20260808 02:16:18.092648 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:16:18.096637 4407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.093336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36096 (local address 127.18.184.190:40287)
0808 02:16:18.093465 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:18.093469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:18.093713 (+ 244us) server_negotiation.cc:407] Connection header received
0808 02:16:18.093803 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:18.093805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:18.093901 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:18.093991 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:18.094248 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.094779 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.095587 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.095848 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.095931 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.096223 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:18.096346 (+ 123us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 02:16:18.096539 (+ 193us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36096: token expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 02:16:18.096743 4407 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36096: token expired
W20260808 02:16:18.097080 3823 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.184.190:40287)
I20260808 02:16:18.105213 4407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.097479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.18.184.190:40287)
0808 02:16:18.097596 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:18.097602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:18.097667 (+ 65us) server_negotiation.cc:407] Connection header received
0808 02:16:18.097815 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:18.097818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:18.097858 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:18.097947 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:18.098574 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.099082 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.099887 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.100119 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.100233 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.101673 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:18.101676 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:18.101679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:18.101713 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:18.103275 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:18.103848 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:18.103853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:18.103854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:18.103948 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:18.104264 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:18.104267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:18.104269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:18.104470 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:18.104630 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:18.104882 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 02:16:18.105018 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:16:18.108323 3906 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:16:18.111881 3906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:16:18.120992 4035 tablet_service.cc:1467] Processing CreateTablet for tablet c888dc4095f84d7383ec0e957e45b5e4 (DEFAULT_TABLE table=test-table [id=9e3566d7ea0645c482a19d3edbbdadf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:18.121820 4035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c888dc4095f84d7383ec0e957e45b5e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:18.125212 3991 tablet_bootstrap.cc:492] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471: Bootstrap starting.
I20260808 02:16:18.126017 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.119318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:43707 (local address 127.0.0.1:39058)
0808 02:16:18.119483 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:18.119499 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:18.119582 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:18.120799 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:18.120802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:18.120822 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:18.120962 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:18.120969 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.122456 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.122462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:18.123783 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:18.123796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.124981 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.124985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:18.125125 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.125211 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 02:16:18.125311 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 02:16:18.125413 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":29}
I20260808 02:16:18.126062 4412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.119701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:39415 (local address 127.0.0.1:47708)
0808 02:16:18.120624 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:18.120642 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:18.120769 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:18.121610 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:18.121613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:18.121635 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:18.121756 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:18.121764 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.123080 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.123086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:18.124209 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:18.124220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.125588 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.125591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:18.125729 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.125754 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:16:18.125834 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 02:16:18.125916 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"mutex_wait_us":1,"thread_start_us":148,"threads_started":1}
I20260808 02:16:18.126559 3991 tablet_bootstrap.cc:654] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:18.126587 4414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.119991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47708 (local address 127.18.184.130:39415)
0808 02:16:18.120549 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:18.120560 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:18.121113 (+ 553us) server_negotiation.cc:407] Connection header received
0808 02:16:18.121243 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:18.121249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:18.121451 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:18.121603 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:18.121894 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.122942 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.124351 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.125258 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.125405 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.125519 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:18.126297 (+ 778us) server_negotiation.cc:299] Negotiation successful
0808 02:16:18.126415 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":152,"threads_started":1}
I20260808 02:16:18.126655 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.119819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39058 (local address 127.18.184.131:43707)
0808 02:16:18.120226 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:18.120246 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:18.120274 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:16:18.120356 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:18.120365 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:18.120623 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:18.120813 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:18.121090 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.122327 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.123939 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.124839 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.124970 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.125114 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:18.125396 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 02:16:18.125503 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":214,"threads_started":1}
I20260808 02:16:18.128232 3991 tablet_bootstrap.cc:492] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471: No bootstrap required, opened a new log
I20260808 02:16:18.128333 3991 ts_tablet_manager.cc:1397] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:16:18.128811 3991 raft_consensus.cc:348] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:18.128947 3991 raft_consensus.cc:374] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:18.128985 3991 raft_consensus.cc:729] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b89f0cc6d3b4856998cc7d081545471, State: Initialized, Role: FOLLOWER
I20260808 02:16:18.129181 3991 consensus_queue.cc:260] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:18.130247 3991 ts_tablet_manager.cc:1428] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:16:18.129935 4333 tablet_service.cc:1467] Processing CreateTablet for tablet c888dc4095f84d7383ec0e957e45b5e4 (DEFAULT_TABLE table=test-table [id=9e3566d7ea0645c482a19d3edbbdadf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:18.130353 4189 tablet_service.cc:1467] Processing CreateTablet for tablet c888dc4095f84d7383ec0e957e45b5e4 (DEFAULT_TABLE table=test-table [id=9e3566d7ea0645c482a19d3edbbdadf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:18.131222 4333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c888dc4095f84d7383ec0e957e45b5e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:18.131393 4189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c888dc4095f84d7383ec0e957e45b5e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:18.138962 4416 tablet_bootstrap.cc:492] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0: Bootstrap starting.
I20260808 02:16:18.139498 4417 tablet_bootstrap.cc:492] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128: Bootstrap starting.
I20260808 02:16:18.142005 4417 tablet_bootstrap.cc:654] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:18.142841 4417 log.cc:824] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:18.143173 4416 tablet_bootstrap.cc:654] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:18.144585 4416 log.cc:824] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:18.145433 4417 tablet_bootstrap.cc:492] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128: No bootstrap required, opened a new log
I20260808 02:16:18.145689 4417 ts_tablet_manager.cc:1397] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260808 02:16:18.147471 4416 tablet_bootstrap.cc:492] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0: No bootstrap required, opened a new log
I20260808 02:16:18.147704 4416 ts_tablet_manager.cc:1397] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260808 02:16:18.149094 4417 raft_consensus.cc:348] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:18.149449 4417 raft_consensus.cc:374] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:18.149539 4417 raft_consensus.cc:729] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 321edff411a349b0860edb469cdb7128, State: Initialized, Role: FOLLOWER
I20260808 02:16:18.150048 4417 consensus_queue.cc:260] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:18.150894 4263 heartbeater.cc:503] Master 127.18.184.190:40287 was elected leader, sending a full tablet report...
I20260808 02:16:18.151202 4417 ts_tablet_manager.cc:1428] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260808 02:16:18.151069 4416 raft_consensus.cc:348] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:18.151360 4416 raft_consensus.cc:374] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:18.151428 4416 raft_consensus.cc:729] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d240237384a4e16a90b2846709f44d0, State: Initialized, Role: FOLLOWER
I20260808 02:16:18.151855 4416 consensus_queue.cc:260] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:18.152778 4403 heartbeater.cc:503] Master 127.18.184.190:40287 was elected leader, sending a full tablet report...
I20260808 02:16:18.152895 4416 ts_tablet_manager.cc:1428] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 02:16:18.280285 4116 raft_consensus.cc:482] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:18.280601 4116 raft_consensus.cc:504] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:18.281934 4116 leader_election.cc:288] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707), 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415)
I20260808 02:16:18.286573 4423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.282288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:39415 (local address 127.18.184.129:52803)
0808 02:16:18.282769 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:18.282784 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:18.282884 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:18.283196 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:18.283198 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:18.283216 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:18.283351 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:18.283358 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.284222 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.284225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:18.285076 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:18.285085 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.285927 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.285931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:18.286040 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.286108 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 02:16:18.286259 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 02:16:18.286362 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":30,"thread_start_us":86,"threads_started":1}
I20260808 02:16:18.286577 4414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.282405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:52803 (local address 127.18.184.130:39415)
0808 02:16:18.282528 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:18.282533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:18.282886 (+ 353us) server_negotiation.cc:407] Connection header received
0808 02:16:18.283004 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:18.283007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:18.283059 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:18.283144 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:18.283485 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.284100 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.285250 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.285763 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.285874 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.285918 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:18.286298 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 02:16:18.286400 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:16:18.286773 4422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.282157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:43707 (local address 127.18.184.129:36261)
0808 02:16:18.282511 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:18.282528 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:18.282662 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:18.282995 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:18.282998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:18.283018 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:18.283183 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:18.283192 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.284140 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.284177 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:18.285287 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:18.285306 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.286311 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.286316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:18.286406 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.286429 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:16:18.286516 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 02:16:18.286593 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":128,"thread_start_us":99,"threads_started":1}
I20260808 02:16:18.286829 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.282350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:36261 (local address 127.18.184.131:43707)
0808 02:16:18.282511 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:18.282517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:18.282617 (+ 100us) server_negotiation.cc:407] Connection header received
0808 02:16:18.282808 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:18.282814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:18.282873 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:18.282953 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:18.283362 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.284002 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.285469 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.286128 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.286259 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.286312 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:18.286569 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 02:16:18.286667 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:16:18.287297 4212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c888dc4095f84d7383ec0e957e45b5e4" candidate_uuid: "4b89f0cc6d3b4856998cc7d081545471" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "321edff411a349b0860edb469cdb7128" is_pre_election: true
I20260808 02:16:18.287515 4353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c888dc4095f84d7383ec0e957e45b5e4" candidate_uuid: "4b89f0cc6d3b4856998cc7d081545471" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d240237384a4e16a90b2846709f44d0" is_pre_election: true
I20260808 02:16:18.287755 4212 raft_consensus.cc:2454] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b89f0cc6d3b4856998cc7d081545471 in term 0.
I20260808 02:16:18.287873 4353 raft_consensus.cc:2454] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b89f0cc6d3b4856998cc7d081545471 in term 0.
I20260808 02:16:18.288271 3986 leader_election.cc:302] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 321edff411a349b0860edb469cdb7128, 4b89f0cc6d3b4856998cc7d081545471; no voters:
I20260808 02:16:18.288607 4116 raft_consensus.cc:2781] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:16:18.288717 4116 raft_consensus.cc:482] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:16:18.288775 4116 raft_consensus.cc:3037] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:18.289938 4116 raft_consensus.cc:504] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:18.290321 4116 leader_election.cc:288] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 1 election: Requested vote from peers 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707), 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415)
I20260808 02:16:18.290733 4353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c888dc4095f84d7383ec0e957e45b5e4" candidate_uuid: "4b89f0cc6d3b4856998cc7d081545471" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d240237384a4e16a90b2846709f44d0"
I20260808 02:16:18.290905 4353 raft_consensus.cc:3037] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:18.290935 4212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c888dc4095f84d7383ec0e957e45b5e4" candidate_uuid: "4b89f0cc6d3b4856998cc7d081545471" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "321edff411a349b0860edb469cdb7128"
I20260808 02:16:18.291062 4212 raft_consensus.cc:3037] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:18.292425 4353 raft_consensus.cc:2454] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b89f0cc6d3b4856998cc7d081545471 in term 1.
I20260808 02:16:18.292538 4212 raft_consensus.cc:2454] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b89f0cc6d3b4856998cc7d081545471 in term 1.
I20260808 02:16:18.292832 3988 leader_election.cc:302] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d240237384a4e16a90b2846709f44d0, 4b89f0cc6d3b4856998cc7d081545471; no voters:
I20260808 02:16:18.293066 4116 raft_consensus.cc:2781] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:18.293372 4116 raft_consensus.cc:686] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 LEADER]: Becoming Leader. State: Replica: 4b89f0cc6d3b4856998cc7d081545471, State: Running, Role: LEADER
I20260808 02:16:18.293655 4116 consensus_queue.cc:237] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:18.296595 3906 catalog_manager.cc:5495] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b89f0cc6d3b4856998cc7d081545471 (127.18.184.129). New cstate: current_term: 1 leader_uuid: "4b89f0cc6d3b4856998cc7d081545471" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } health_report { overall_health: HEALTHY } } }
W20260808 02:16:18.321204 4404 tablet.cc:2369] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:16:18.322765 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.318672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55744 (local address 127.18.184.129:44709)
0808 02:16:18.319021 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:18.319028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:18.319039 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:16:18.319119 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:18.319122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:18.319176 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:18.319321 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:18.319590 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.320120 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.320976 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:18.321280 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:18.321407 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:18.321772 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:18.322245 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:16:18.322348 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:18.322468 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 02:16:18.322527 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":128,"threads_started":1}
I20260808 02:16:18.327481 4212 raft_consensus.cc:1261] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128 [term 1 FOLLOWER]: Refusing update from remote peer 4b89f0cc6d3b4856998cc7d081545471: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:18.327481 4353 raft_consensus.cc:1261] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0 [term 1 FOLLOWER]: Refusing update from remote peer 4b89f0cc6d3b4856998cc7d081545471: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:18.328384 4116 consensus_queue.cc:1035] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:18.328670 4424 consensus_queue.cc:1035] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [LEADER]: Connected to new peer: Peer: permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:18.340700 4428 mvcc.cc:205] Tried to move back new op lower bound from 7316092429621821440 to 7316092429493194752. Current Snapshot: MvccSnapshot[applied={T|T < 7316092429621821440 or (T in {7316092429621821440})}]
I20260808 02:16:18.340930 4430 mvcc.cc:205] Tried to move back new op lower bound from 7316092429621821440 to 7316092429493194752. Current Snapshot: MvccSnapshot[applied={T|T < 7316092429621821440 or (T in {7316092429621821440})}]
I20260808 02:16:18.346612 3906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:18.346902 3906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:18.349382 3906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Sending DeleteTablet for 3 replicas of tablet c888dc4095f84d7383ec0e957e45b5e4
I20260808 02:16:18.350252 4035 tablet_service.cc:1514] Processing DeleteTablet for tablet c888dc4095f84d7383ec0e957e45b5e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:18 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:55740
I20260808 02:16:18.350196 4333 tablet_service.cc:1514] Processing DeleteTablet for tablet c888dc4095f84d7383ec0e957e45b5e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:18 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:39058
I20260808 02:16:18.350317 4189 tablet_service.cc:1514] Processing DeleteTablet for tablet c888dc4095f84d7383ec0e957e45b5e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:18 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:47708
I20260808 02:16:18.350754 4439 tablet_replica.cc:330] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0: stopping tablet replica
I20260808 02:16:18.350983 4440 tablet_replica.cc:330] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128: stopping tablet replica
I20260808 02:16:18.351145 4439 raft_consensus.cc:2229] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:18.351161 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 3947
I20260808 02:16:18.351501 4439 raft_consensus.cc:2258] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:18.352851 4440 raft_consensus.cc:2229] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:18.353175 4440 raft_consensus.cc:2258] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:18.355239 4440 ts_tablet_manager.cc:1905] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:18.355340 4439 ts_tablet_manager.cc:1905] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:18.358283 4440 ts_tablet_manager.cc:1918] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:18.358283 4439 ts_tablet_manager.cc:1918] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:18.358409 4439 log.cc:1196] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/c888dc4095f84d7383ec0e957e45b5e4
I20260808 02:16:18.358409 4440 log.cc:1196] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/c888dc4095f84d7383ec0e957e45b5e4
I20260808 02:16:18.358798 4440 ts_tablet_manager.cc:1939] T c888dc4095f84d7383ec0e957e45b5e4 P 321edff411a349b0860edb469cdb7128: Deleting consensus metadata
I20260808 02:16:18.358886 4439 ts_tablet_manager.cc:1939] T c888dc4095f84d7383ec0e957e45b5e4 P 3d240237384a4e16a90b2846709f44d0: Deleting consensus metadata
I20260808 02:16:18.360348 3888 catalog_manager.cc:4841] TS 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415): tablet c888dc4095f84d7383ec0e957e45b5e4 (table test-table [id=9e3566d7ea0645c482a19d3edbbdadf5]) successfully deleted
I20260808 02:16:18.360426 3890 catalog_manager.cc:4841] TS 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707): tablet c888dc4095f84d7383ec0e957e45b5e4 (table test-table [id=9e3566d7ea0645c482a19d3edbbdadf5]) successfully deleted
W20260808 02:16:18.362206 3888 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:44709 (error 108)
I20260808 02:16:18.362665 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 4114
I20260808 02:16:18.365798 4412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:18.363067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:44709 (local address 127.0.0.1:55754)
0808 02:16:18.365573 (+ 2506us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:18.365670 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2378}
W20260808 02:16:18.366436 3888 catalog_manager.cc:4584] TS 4b89f0cc6d3b4856998cc7d081545471 (127.18.184.129:44709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c888dc4095f84d7383ec0e957e45b5e4: Network error: Client connection negotiation failed: client connection to 127.18.184.129:44709: connect: Connection refused (error 111)
I20260808 02:16:18.372382 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 4266
I20260808 02:16:18.381984 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 3869
I20260808 02:16:38.391316 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:40287
--webserver_interface=127.18.184.190
--webserver_port=34609
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:40287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:38.511439 4442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:38.511796 4442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:38.511886 4442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:38.516134 4442 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 02:16:38.516229 4442 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 02:16:38.516253 4442 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 02:16:38.516276 4442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:38.516323 4442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:38.516364 4442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:38.516398 4442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:38.516433 4442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:38.522100 4442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:40287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:40287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=34609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:38.523288 4442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:38.524367 4442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:38.533850 4448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:38.533921 4442 server_base.cc:1034] running on GCE node
W20260808 02:16:38.533936 4447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:38.533962 4450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:38.534888 4442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:38.536082 4442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:38.537235 4442 hybrid_clock.cc:648] HybridClock initialized: now 1786155398537211 us; error 29 us; skew 500 ppm
Aug 08 02:16:38 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:38.540868 4442 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:16:38.541945 4442 webserver.cc:460] Webserver started at http://127.18.184.190:34609/ using document root <none> and password file <none>
I20260808 02:16:38.542572 4442 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:38.542641 4442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:38.546952 4442 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:16:38.549727 4458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:38.551357 4442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:38.551582 4442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "a4228483587740a1a8333148c6b43e66"
format_stamp: "Formatted at 2026-08-08 02:15:55 on dist-test-slave-9gcw"
server_key: "7759432198d95601bc323e16a15151a5"
server_key_iv: "1f6f745e5d893c1fd826ebe4e00b2f76"
server_key_version: "encryptionkey@0"
I20260808 02:16:38.552233 4442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:38.574514 4442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:38.578473 4442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:38.578729 4442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:38.586783 4442 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:40287
I20260808 02:16:38.587107 4518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:40287 every 8 connection(s)
I20260808 02:16:38.587926 4442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:38.591084 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 4442
I20260808 02:16:38.591629 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:44709
--local_ip_for_outbound_sockets=127.18.184.129
--tserver_master_addrs=127.18.184.190:40287
--webserver_port=39235
--webserver_interface=127.18.184.129
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:16:38.594523 4519 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:16:38.596728 4519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Bootstrap starting.
I20260808 02:16:38.617095 4519 log.cc:824] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:38.630059 4519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:16:38.630569 4519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Bootstrap complete.
I20260808 02:16:38.635295 4519 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } }
I20260808 02:16:38.635850 4519 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4228483587740a1a8333148c6b43e66, State: Initialized, Role: FOLLOWER
I20260808 02:16:38.636430 4519 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } }
I20260808 02:16:38.636631 4519 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:38.636732 4519 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:38.636866 4519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 2 FOLLOWER]: Advancing to term 3
I20260808 02:16:38.639356 4519 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } }
I20260808 02:16:38.639767 4519 leader_election.cc:302] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4228483587740a1a8333148c6b43e66; no voters:
I20260808 02:16:38.640228 4519 leader_election.cc:288] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 02:16:38.640331 4523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 02:16:38.640568 4523 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [term 3 LEADER]: Becoming Leader. State: Replica: a4228483587740a1a8333148c6b43e66, State: Running, Role: LEADER
I20260808 02:16:38.640914 4523 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } }
I20260808 02:16:38.641708 4519 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:38.642033 4523 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4228483587740a1a8333148c6b43e66. Latest consensus state: current_term: 3 leader_uuid: "a4228483587740a1a8333148c6b43e66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } } }
I20260808 02:16:38.642174 4523 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:38.642583 4527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:38.642580 4524 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a4228483587740a1a8333148c6b43e66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4228483587740a1a8333148c6b43e66" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 40287 } } }
I20260808 02:16:38.642907 4524 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:38.648766 4527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:38.649593 4527 catalog_manager.cc:1259] Loaded cluster ID: ad53c1bc050d4f99b6dde1ab60fee4bd
I20260808 02:16:38.649698 4527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:38.651713 4527 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:38.652420 4527 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Loaded TSK: 0
I20260808 02:16:38.652527 4527 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a4228483587740a1a8333148c6b43e66: Loaded TSK: 1
F20260808 02:16:38.653409 4527 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7468bc29ad google::LogMessage::Fail() at ??:0
@ 0x7f7468bc588a google::LogMessage::SendToLog() at ??:0
@ 0x7f7468bc24de google::LogMessage::Flush() at ??:0
@ 0x7f7468bc3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f746b88618d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f746b87ef0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f747352b7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f747350599c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f7473506178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f7473547df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f747356682b std::function<>::operator()() at ??:0
@ 0x7f7473505b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f7473506f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7473503579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f7473547640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f746ae143d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f746ae14c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f746ae163f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f746ae05512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f746c4496db start_thread at ??:0
@ 0x7f74676c061f clone at ??:0
W20260808 02:16:38.761125 4521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:38.761571 4521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:38.761651 4521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:38.769273 4521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:38.769356 4521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:38.769459 4521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:16:38.779176 4521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:44709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=39235
--tserver_master_addrs=127.18.184.190:40287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:38.780862 4521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:38.782374 4521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:38.794567 4576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:38.794685 4571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:38.794560 4570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:38.795707 4521 server_base.cc:1034] running on GCE node
I20260808 02:16:38.796329 4521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:38.797315 4521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:38.798548 4521 hybrid_clock.cc:648] HybridClock initialized: now 1786155398798512 us; error 55 us; skew 500 ppm
Aug 08 02:16:38 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:38.802289 4521 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:16:38.803304 4521 webserver.cc:460] Webserver started at http://127.18.184.129:39235/ using document root <none> and password file <none>
I20260808 02:16:38.803895 4521 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:38.803960 4521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:38.808311 4521 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260808 02:16:38.810846 4583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:38.812265 4521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:16:38.812430 4521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "4b89f0cc6d3b4856998cc7d081545471"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "baba4c5d31da52385d14ddb4ec59ea6c"
server_key_iv: "96386123645954c31047f5c0386a110c"
server_key_version: "encryptionkey@0"
I20260808 02:16:38.813077 4521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:38.831590 4521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:38.838270 4521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:38.839315 4521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:38.842381 4521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:38.844058 4590 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:16:38.854271 4521 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:16:38.854390 4521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.011s user 0.001s sys 0.000s
I20260808 02:16:38.854486 4521 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:16:38.856422 4521 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:16:38.856528 4521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 02:16:38.856876 4590 tablet_bootstrap.cc:492] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471: Bootstrap starting.
I20260808 02:16:38.868175 4521 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:44709
I20260808 02:16:38.868484 4708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:44709 every 8 connection(s)
I20260808 02:16:38.869411 4521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:16:38.872617 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 4521
I20260808 02:16:38.873113 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:39415
--local_ip_for_outbound_sockets=127.18.184.130
--tserver_master_addrs=127.18.184.190:40287
--webserver_port=41967
--webserver_interface=127.18.184.130
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:16:38.883459 4590 log.cc:824] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:38.900691 4590 tablet_bootstrap.cc:492] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:16:38.902036 4590 tablet_bootstrap.cc:492] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471: Bootstrap complete.
I20260808 02:16:38.906908 4590 ts_tablet_manager.cc:1397] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471: Time spent bootstrapping tablet: real 0.050s user 0.032s sys 0.014s
I20260808 02:16:38.921509 4590 raft_consensus.cc:348] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:38.924943 4590 raft_consensus.cc:729] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b89f0cc6d3b4856998cc7d081545471, State: Initialized, Role: FOLLOWER
I20260808 02:16:38.927587 4590 consensus_queue.cc:260] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:38.928973 4590 ts_tablet_manager.cc:1428] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471: Time spent starting tablet: real 0.022s user 0.018s sys 0.005s
W20260808 02:16:39.085001 4711 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:16:38.871572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:51795)
0808 02:16:38.872120 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:38.872186 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:38.874606 (+ 2420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:39.084181 (+209575us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":154,"threads_started":1}
W20260808 02:16:39.085755 4709 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:40287 (0 consecutive failures): Network error: Failed to ping master at 127.18.184.190:40287: Client connection negotiation failed: client connection to 127.18.184.190:40287: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 02:16:39.086592 4711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:39.086311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:57847)
0808 02:16:39.086447 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:39.086528 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 02:16:39.087373 4711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:39.087153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:38287)
0808 02:16:39.087264 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:39.087311 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 02:16:39.087595 4709 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:16:39.139278 4712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:39.139611 4712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:39.139715 4712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:39.144454 4712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:39.144569 4712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:39.144687 4712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:16:39.150393 4712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:39415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=41967
--tserver_master_addrs=127.18.184.190:40287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:39.151629 4712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:39.152701 4712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:39.163180 4722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:39.163257 4724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:39.163192 4721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:39.163565 4712 server_base.cc:1034] running on GCE node
I20260808 02:16:39.164022 4712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:39.164691 4712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:39.165858 4712 hybrid_clock.cc:648] HybridClock initialized: now 1786155399165844 us; error 28 us; skew 500 ppm
Aug 08 02:16:39 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155399, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:39.168736 4712 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:16:39.169667 4712 webserver.cc:460] Webserver started at http://127.18.184.130:41967/ using document root <none> and password file <none>
I20260808 02:16:39.170238 4712 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:39.170308 4712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:39.174325 4712 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:16:39.176468 4732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:39.177846 4712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:39.177949 4712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "321edff411a349b0860edb469cdb7128"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "a82621d6a047664c44e31da6f941fdef"
server_key_iv: "2f5dc4e913ad1af023d3f39023ef8f00"
server_key_version: "encryptionkey@0"
I20260808 02:16:39.178505 4712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:39.205739 4712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:39.208884 4712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:39.209060 4712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:39.209743 4712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:39.210644 4712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:39.210721 4712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:39.210788 4712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:39.210831 4712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:39.220963 4712 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:39415
I20260808 02:16:39.221006 4855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:39415 every 8 connection(s)
I20260808 02:16:39.222067 4712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:16:39.222498 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 4712
I20260808 02:16:39.223089 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:43707
--local_ip_for_outbound_sockets=127.18.184.131
--tserver_master_addrs=127.18.184.190:40287
--webserver_port=43573
--webserver_interface=127.18.184.131
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:16:39.226547 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:39.224487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:54051)
0808 02:16:39.225006 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:39.225141 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":148,"threads_started":1}
W20260808 02:16:39.227167 4856 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:40287 (0 consecutive failures): Network error: Failed to ping master at 127.18.184.190:40287: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
I20260808 02:16:39.227950 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:39.227665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:44071)
0808 02:16:39.227841 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:39.227892 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 02:16:39.228765 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:39.228500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:50981)
0808 02:16:39.228653 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:39.228711 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260808 02:16:39.228988 4856 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:16:39.343397 4859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:39.343689 4859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:39.343808 4859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:39.348121 4859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:39.348194 4859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:39.348290 4859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:16:39.353983 4859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:43707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=43573
--tserver_master_addrs=127.18.184.190:40287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:39.355196 4859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:39.356312 4859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:39.367093 4866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:39.367040 4867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:39.367182 4859 server_base.cc:1034] running on GCE node
W20260808 02:16:39.367064 4869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:39.367966 4859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:39.368770 4859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:39.369992 4859 hybrid_clock.cc:648] HybridClock initialized: now 1786155399369960 us; error 51 us; skew 500 ppm
Aug 08 02:16:39 dist-test-slave-9gcw krb5kdc[3264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155399, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:39.372985 4859 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:16:39.374024 4859 webserver.cc:460] Webserver started at http://127.18.184.131:43573/ using document root <none> and password file <none>
I20260808 02:16:39.374610 4859 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:39.374703 4859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:39.378742 4859 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:16:39.380961 4876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:39.382505 4859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:39.382660 4859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "3d240237384a4e16a90b2846709f44d0"
format_stamp: "Formatted at 2026-08-08 02:15:56 on dist-test-slave-9gcw"
server_key: "b6508d4a13f74fc3b376fc8830f78e58"
server_key_iv: "2c16932f9be1213895f4911d580b45ee"
server_key_version: "encryptionkey@0"
I20260808 02:16:39.383270 4859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:39.396996 4859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:39.400538 4859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:39.400777 4859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:39.401489 4859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:39.402365 4859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:39.402437 4859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:39.402510 4859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:39.402563 4859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:39.412076 4859 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:43707
I20260808 02:16:39.412159 5007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:43707 every 8 connection(s)
I20260808 02:16:39.413208 4859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:16:39.417398 5010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:39.415433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:43053)
0808 02:16:39.415918 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:39.416030 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":116,"threads_started":1}
W20260808 02:16:39.418013 5008 heartbeater.cc:644] Failed to heartbeat to 127.18.184.190:40287 (0 consecutive failures): Network error: Failed to ping master at 127.18.184.190:40287: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
I20260808 02:16:39.418635 5010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:39.418418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:35031)
0808 02:16:39.418530 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:39.418579 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 02:16:39.419261 5010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:39.419062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:33909)
0808 02:16:39.419163 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:39.419204 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 02:16:39.419454 5008 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:16:39.422492 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 4859
I20260808 02:16:40.088881 5012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:40.088309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:52741)
0808 02:16:40.088600 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:40.088702 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":103,"threads_started":1}
I20260808 02:16:40.230141 5013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:40.229624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:55101)
0808 02:16:40.229936 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:40.230014 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":78,"threads_started":1}
I20260808 02:16:40.266669 5014 raft_consensus.cc:482] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:40.266878 5014 raft_consensus.cc:504] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:40.268117 5014 leader_election.cc:288] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707), 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415)
Aug 08 02:16:40 dist-test-slave-9gcw krb5kdc[3264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.130@KRBTEST.COM
Aug 08 02:16:40 dist-test-slave-9gcw krb5kdc[3264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.131@KRBTEST.COM
I20260808 02:16:40.284399 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:40.268810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:49527 (local address 127.18.184.130:39415)
0808 02:16:40.269276 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:40.269300 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:40.269339 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:16:40.270167 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:40.270174 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:40.270503 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:40.270894 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:40.272055 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:40.273151 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:40.274583 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:40.275162 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:40.275382 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:40.278881 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:40.278891 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:40.278913 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:40.278961 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:40.281033 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:40.281848 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:40.281858 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:40.281864 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:40.282049 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:40.282577 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:40.282583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:40.282586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:40.283125 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:40.283242 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:40.283901 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 02:16:40.284219 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":65,"threads_started":1}
I20260808 02:16:40.284390 5015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:40.268469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:43707 (local address 127.18.184.129:38795)
0808 02:16:40.268964 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:40.268982 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:40.269077 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:40.270785 (+ 1708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:40.270797 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:40.271395 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:40.271840 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:40.271856 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:40.273273 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:40.273281 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:40.274433 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:40.274446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:40.275241 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:40.275248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:40.275513 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:40.276346 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:40.276362 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:40.278831 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:40.281478 (+ 2647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:40.281481 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:40.281482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:40.281903 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:40.282396 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:40.282398 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:40.282400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:40.282480 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:40.283336 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:40.283339 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:40.283561 (+ 222us) client_negotiation.cc:769] Sending connection context
0808 02:16:40.283806 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 02:16:40.284128 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":209,"spinlock_wait_cycles":25216,"thread_start_us":103,"threads_started":1}
I20260808 02:16:40.284518 5016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:40.268641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:38795 (local address 127.18.184.131:43707)
0808 02:16:40.269268 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:40.269280 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:40.269319 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:16:40.269985 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:40.269991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:40.270361 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:40.270629 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:40.272057 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:40.273058 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:40.274611 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:40.275090 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:40.275220 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:40.279059 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:40.279064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:40.279074 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:40.279106 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:40.281311 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:40.282055 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:40.282061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:40.282065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:40.282283 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:40.282607 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:40.282613 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:40.282620 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:40.283211 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:40.283361 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:40.283982 (+ 621us) server_negotiation.cc:299] Negotiation successful
0808 02:16:40.284300 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":209,"threads_started":1}
I20260808 02:16:40.284495 5012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:40.268469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:39415 (local address 127.18.184.129:49527)
0808 02:16:40.268762 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:40.268782 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:40.268918 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:40.270859 (+ 1941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:40.270862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:40.271370 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:40.271840 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:40.271855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:40.273339 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:40.273341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:40.274434 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:40.274445 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:40.275334 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:40.275337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:40.275515 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:40.276304 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:40.276320 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:40.278574 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:40.281186 (+ 2612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:40.281194 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:40.281212 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:40.281621 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:40.282204 (+ 583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:40.282207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:40.282209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:40.282470 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:40.283273 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:40.283284 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:40.283568 (+ 284us) client_negotiation.cc:769] Sending connection context
0808 02:16:40.283731 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 02:16:40.284252 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":364,"spinlock_wait_cycles":67456}
W20260808 02:16:40.288215 4585 leader_election.cc:341] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415): Not found: Tablet not found: c888dc4095f84d7383ec0e957e45b5e4
W20260808 02:16:40.288650 4587 leader_election.cc:341] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707): Not found: Tablet not found: c888dc4095f84d7383ec0e957e45b5e4
I20260808 02:16:40.288745 4587 leader_election.cc:302] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b89f0cc6d3b4856998cc7d081545471; no voters: 321edff411a349b0860edb469cdb7128, 3d240237384a4e16a90b2846709f44d0
I20260808 02:16:40.289130 5014 raft_consensus.cc:2726] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:16:40.420559 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:40.420085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:49155)
0808 02:16:40.420385 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:40.420451 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":85,"threads_started":1}
I20260808 02:16:41.090415 5019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:41.089850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:51505)
0808 02:16:41.090206 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:41.090281 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
I20260808 02:16:41.231453 5020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:41.231002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:42071)
0808 02:16:41.231281 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:41.231357 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":75,"threads_started":1}
I20260808 02:16:41.421878 5021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:41.421415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:51775)
0808 02:16:41.421695 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:41.421759 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":75,"threads_started":1}
I20260808 02:16:41.944085 5022 raft_consensus.cc:482] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:41.944252 5022 raft_consensus.cc:504] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:41.944774 5022 leader_election.cc:288] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707), 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415)
W20260808 02:16:41.945842 4587 leader_election.cc:341] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707): Not found: Tablet not found: c888dc4095f84d7383ec0e957e45b5e4
W20260808 02:16:41.945966 4585 leader_election.cc:341] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415): Not found: Tablet not found: c888dc4095f84d7383ec0e957e45b5e4
I20260808 02:16:41.946079 4585 leader_election.cc:302] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b89f0cc6d3b4856998cc7d081545471; no voters: 321edff411a349b0860edb469cdb7128, 3d240237384a4e16a90b2846709f44d0
I20260808 02:16:41.946305 5022 raft_consensus.cc:2726] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:16:42.091950 5023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.091447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:56841)
0808 02:16:42.091785 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:42.091842 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":89,"threads_started":1}
I20260808 02:16:42.232834 5024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.232359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:34153)
0808 02:16:42.232662 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:42.232737 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":90,"threads_started":1}
I20260808 02:16:42.423245 5025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.422721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:49393)
0808 02:16:42.423040 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:42.423106 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":97,"threads_started":1}
I20260808 02:16:43.093320 5026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.092818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:46981)
0808 02:16:43.093112 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:43.093195 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":95,"threads_started":1}
I20260808 02:16:43.234232 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.233684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:32997)
0808 02:16:43.234037 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:43.234101 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":101,"threads_started":1}
I20260808 02:16:43.424512 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.424069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:58055)
0808 02:16:43.424357 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:43.424415 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":92,"threads_started":1}
I20260808 02:16:44.084935 5029 raft_consensus.cc:482] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:44.085109 5029 raft_consensus.cc:504] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:44.085721 5029 leader_election.cc:288] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707), 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415)
W20260808 02:16:44.086591 4587 leader_election.cc:341] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707): Not found: Tablet not found: c888dc4095f84d7383ec0e957e45b5e4
W20260808 02:16:44.086737 4585 leader_election.cc:341] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415): Not found: Tablet not found: c888dc4095f84d7383ec0e957e45b5e4
I20260808 02:16:44.086835 4585 leader_election.cc:302] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b89f0cc6d3b4856998cc7d081545471; no voters: 321edff411a349b0860edb469cdb7128, 3d240237384a4e16a90b2846709f44d0
I20260808 02:16:44.086963 5029 raft_consensus.cc:2726] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:16:44.094728 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:44.094268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:43309)
0808 02:16:44.094572 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:44.094631 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":94,"threads_started":1}
I20260808 02:16:44.235582 5031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:44.235100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:50479)
0808 02:16:44.235418 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:44.235480 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":81,"threads_started":1}
I20260808 02:16:44.425835 5032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:44.425366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:56713)
0808 02:16:44.425655 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:44.425727 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":95,"threads_started":1}
I20260808 02:16:45.096051 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.095578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:48989)
0808 02:16:45.095886 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:45.095948 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":101,"threads_started":1}
I20260808 02:16:45.237085 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.236536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:43165)
0808 02:16:45.236897 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:45.236957 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":156,"threads_started":1}
I20260808 02:16:45.427119 5035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.426596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:33491)
0808 02:16:45.426884 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:45.426938 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":96,"threads_started":1}
I20260808 02:16:46.097718 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.097132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:47573)
0808 02:16:46.097465 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:46.097549 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":79,"threads_started":1}
I20260808 02:16:46.238750 5037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.238110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:38211)
0808 02:16:46.238550 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:46.238628 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":95,"threads_started":1}
I20260808 02:16:46.428566 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.428047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:52575)
0808 02:16:46.428413 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:46.428467 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":136,"threads_started":1}
I20260808 02:16:47.099051 5041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:47.098559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:55501)
0808 02:16:47.098869 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:47.098935 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":84,"threads_started":1}
I20260808 02:16:47.240046 5042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:47.239604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:42833)
0808 02:16:47.239884 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:47.239942 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":102,"threads_started":1}
I20260808 02:16:47.429857 5043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:47.429420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:41211)
0808 02:16:47.429689 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:47.429757 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
I20260808 02:16:48.100499 5045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.099998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:49905)
0808 02:16:48.100318 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:48.100389 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":86,"threads_started":1}
I20260808 02:16:48.241461 5046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.240910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:59971)
0808 02:16:48.241294 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:48.241360 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":98,"threads_started":1}
I20260808 02:16:48.379727 5047 raft_consensus.cc:482] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:48.379933 5047 raft_consensus.cc:504] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:48.380416 5047 leader_election.cc:288] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707), 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415)
W20260808 02:16:48.381306 4587 leader_election.cc:341] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707): Not found: Tablet not found: c888dc4095f84d7383ec0e957e45b5e4
W20260808 02:16:48.381428 4585 leader_election.cc:341] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415): Not found: Tablet not found: c888dc4095f84d7383ec0e957e45b5e4
I20260808 02:16:48.381541 4585 leader_election.cc:302] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b89f0cc6d3b4856998cc7d081545471; no voters: 321edff411a349b0860edb469cdb7128, 3d240237384a4e16a90b2846709f44d0
I20260808 02:16:48.381681 5047 raft_consensus.cc:2726] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:16:48.431195 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.430688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:42881)
0808 02:16:48.431011 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:48.431080 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":108,"threads_started":1}
I20260808 02:16:49.102082 5049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.101613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:55001)
0808 02:16:49.101919 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:49.101980 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":82,"threads_started":1}
I20260808 02:16:49.242929 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.242502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:50483)
0808 02:16:49.242761 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:49.242826 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":75,"threads_started":1}
I20260808 02:16:49.432427 5051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.432015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:52219)
0808 02:16:49.432272 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:49.432326 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":74,"threads_started":1}
I20260808 02:16:50.103379 5052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:50.102900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:44585)
0808 02:16:50.103186 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:50.103270 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":76,"threads_started":1}
I20260808 02:16:50.244220 5053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:50.243792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:32781)
0808 02:16:50.244062 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:50.244118 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
I20260808 02:16:50.433820 5054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:50.433358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:53453)
0808 02:16:50.433625 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:50.433681 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":78,"threads_started":1}
I20260808 02:16:51.104625 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:51.104195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:38869)
0808 02:16:51.104455 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:51.104515 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":84,"threads_started":1}
I20260808 02:16:51.245628 5056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:51.245123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:41281)
0808 02:16:51.245456 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:51.245524 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":80,"threads_started":1}
I20260808 02:16:51.435096 5057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:51.434656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:37065)
0808 02:16:51.434931 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:51.434992 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":83,"threads_started":1}
I20260808 02:16:52.106068 5058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:52.105605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:43037)
0808 02:16:52.105894 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:52.105960 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":88,"threads_started":1}
I20260808 02:16:52.246872 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:52.246416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:56281)
0808 02:16:52.246681 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:52.246740 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":84,"threads_started":1}
I20260808 02:16:52.436597 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:52.436102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:34447)
0808 02:16:52.436417 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:52.436486 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":84,"threads_started":1}
I20260808 02:16:52.595907 5062 raft_consensus.cc:482] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:52.596062 5062 raft_consensus.cc:504] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d240237384a4e16a90b2846709f44d0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 43707 } } peers { permanent_uuid: "321edff411a349b0860edb469cdb7128" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 39415 } } peers { permanent_uuid: "4b89f0cc6d3b4856998cc7d081545471" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 44709 } }
I20260808 02:16:52.596554 5062 leader_election.cc:288] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707), 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415)
W20260808 02:16:52.597441 4587 leader_election.cc:341] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d240237384a4e16a90b2846709f44d0 (127.18.184.131:43707): Not found: Tablet not found: c888dc4095f84d7383ec0e957e45b5e4
W20260808 02:16:52.597575 4585 leader_election.cc:341] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 321edff411a349b0860edb469cdb7128 (127.18.184.130:39415): Not found: Tablet not found: c888dc4095f84d7383ec0e957e45b5e4
I20260808 02:16:52.597666 4587 leader_election.cc:302] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b89f0cc6d3b4856998cc7d081545471; no voters: 321edff411a349b0860edb469cdb7128, 3d240237384a4e16a90b2846709f44d0
I20260808 02:16:52.597859 5062 raft_consensus.cc:2726] T c888dc4095f84d7383ec0e957e45b5e4 P 4b89f0cc6d3b4856998cc7d081545471 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:16:53.107396 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:53.106938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:53289)
0808 02:16:53.107224 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:53.107288 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":90,"threads_started":1}
I20260808 02:16:53.248500 5064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:53.247890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:38531)
0808 02:16:53.248269 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:53.248343 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":127,"threads_started":1}
I20260808 02:16:53.438118 5065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:53.437625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:53149)
0808 02:16:53.437917 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:53.438004 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":105,"threads_started":1}
I20260808 02:16:54.108884 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:54.108443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.129:52407)
0808 02:16:54.108719 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:54.108781 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":86,"threads_started":1}
I20260808 02:16:54.250154 5067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:54.249613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.130:57307)
0808 02:16:54.249981 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:54.250050 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":83,"threads_started":1}
I20260808 02:16:54.427345 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 4521
I20260808 02:16:54.436862 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 4712
I20260808 02:16:54.441897 5068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:54.440823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:40287 (local address 127.18.184.131:51831)
0808 02:16:54.441739 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:54.441801 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.190:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":826,"thread_start_us":191,"threads_started":1}
I20260808 02:16:54.447530 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 4859
2026-08-08T02:16:54Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60032 ms)
[----------] 30 tests from SecurityITest (252603 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:54 dist-test-slave-9gcw krb5kdc[5071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:54 dist-test-slave-9gcw krb5kdc[5071](info): set up 2 sockets
Aug 08 02:16:54 dist-test-slave-9gcw krb5kdc[5071](info): commencing operation
krb5kdc: starting...
W20260808 02:16:55.600975 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:16:55 dist-test-slave-9gcw krb5kdc[5071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:16:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:16:55.748329 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:46717
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:39831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:46717
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:55.865808 5086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:55.866197 5086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:55.866285 5086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:55.870551 5086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:55.870668 5086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:55.870710 5086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:55.870767 5086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:55.870827 5086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:55.876739 5086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:39831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:46717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:46717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:55.878007 5086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:55.879276 5086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:55.889096 5092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:55.889192 5094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:55.889096 5091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:55.889537 5086 server_base.cc:1034] running on GCE node
I20260808 02:16:55.890261 5086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:55.891553 5086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:55.892777 5086 hybrid_clock.cc:648] HybridClock initialized: now 1786155415892760 us; error 36 us; skew 500 ppm
Aug 08 02:16:55 dist-test-slave-9gcw krb5kdc[5071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155415, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:55.896061 5086 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:16:55.897259 5086 webserver.cc:460] Webserver started at http://127.18.184.190:45995/ using document root <none> and password file <none>
I20260808 02:16:55.897921 5086 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:55.897994 5086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:55.898252 5086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:55.900673 5086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "3d4bc6d35b814be580e0dfc0f7d1fc93"
format_stamp: "Formatted at 2026-08-08 02:16:55 on dist-test-slave-9gcw"
server_key: "5f8117449ea9f0092065417d190bed22"
server_key_iv: "ade00b0b63b3707038debfafaad6438e"
server_key_version: "encryptionkey@0"
I20260808 02:16:55.901363 5086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "3d4bc6d35b814be580e0dfc0f7d1fc93"
format_stamp: "Formatted at 2026-08-08 02:16:55 on dist-test-slave-9gcw"
server_key: "5f8117449ea9f0092065417d190bed22"
server_key_iv: "ade00b0b63b3707038debfafaad6438e"
server_key_version: "encryptionkey@0"
I20260808 02:16:55.905893 5086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:16:55.909088 5102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:55.911162 5086 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:16:55.911369 5086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "3d4bc6d35b814be580e0dfc0f7d1fc93"
format_stamp: "Formatted at 2026-08-08 02:16:55 on dist-test-slave-9gcw"
server_key: "5f8117449ea9f0092065417d190bed22"
server_key_iv: "ade00b0b63b3707038debfafaad6438e"
server_key_version: "encryptionkey@0"
I20260808 02:16:55.911556 5086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:55.940359 5086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:55.943650 5086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:55.943876 5086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:55.951347 5086 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:46717
I20260808 02:16:55.951375 5158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:46717 every 8 connection(s)
I20260808 02:16:55.952356 5086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:55.955367 5159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:55.956849 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 5086
I20260808 02:16:55.956998 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:16:55.957382 19170 external_mini_cluster.cc:1428] Setting key 75ab3d6eb483da230a4f6b573321c708
I20260808 02:16:55.962034 5159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93: Bootstrap starting.
I20260808 02:16:55.964792 5159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:55.965706 5159 log.cc:824] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:16:55 dist-test-slave-9gcw krb5kdc[5071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:16:55.967895 5159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93: No bootstrap required, opened a new log
I20260808 02:16:55.972175 5159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d4bc6d35b814be580e0dfc0f7d1fc93" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46717 } }
I20260808 02:16:55.972468 5159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:55.972530 5159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d4bc6d35b814be580e0dfc0f7d1fc93, State: Initialized, Role: FOLLOWER
I20260808 02:16:55.972591 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:55.958819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41534 (local address 127.18.184.190:46717)
0808 02:16:55.959240 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:55.959257 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:55.959294 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:16:55.959943 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:55.959959 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:55.960298 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:55.960581 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:55.961621 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:55.962749 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:55.963963 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:55.964434 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:55.964557 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:55.967377 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:55.967381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:55.967395 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:55.967421 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:55.969277 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:55.970024 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:55.970032 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:55.970044 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:55.970161 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:55.970463 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:55.970468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:55.970469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:55.970880 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:55.971102 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:55.971385 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 02:16:55.971707 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":127,"threads_started":1}
I20260808 02:16:55.973057 5159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d4bc6d35b814be580e0dfc0f7d1fc93" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46717 } }
I20260808 02:16:55.973264 5159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:55.973336 5159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:55.973438 5159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:55.974383 5159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d4bc6d35b814be580e0dfc0f7d1fc93" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46717 } }
I20260808 02:16:55.974741 5159 leader_election.cc:302] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d4bc6d35b814be580e0dfc0f7d1fc93; no voters:
I20260808 02:16:55.975075 5159 leader_election.cc:288] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:55.975194 5164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:55.975469 5164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [term 1 LEADER]: Becoming Leader. State: Replica: 3d4bc6d35b814be580e0dfc0f7d1fc93, State: Running, Role: LEADER
I20260808 02:16:55.975816 5164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d4bc6d35b814be580e0dfc0f7d1fc93" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46717 } }
I20260808 02:16:55.976338 5159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:55.977744 5165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d4bc6d35b814be580e0dfc0f7d1fc93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d4bc6d35b814be580e0dfc0f7d1fc93" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46717 } } }
I20260808 02:16:55.977905 5165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:55.977743 5166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d4bc6d35b814be580e0dfc0f7d1fc93. Latest consensus state: current_term: 1 leader_uuid: "3d4bc6d35b814be580e0dfc0f7d1fc93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d4bc6d35b814be580e0dfc0f7d1fc93" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46717 } } }
I20260808 02:16:55.978235 5166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:55.978354 5172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:55.981194 5172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:55.986608 5172 catalog_manager.cc:1347] Generated new cluster ID: 1e87ea2330f54df1a86b907043862e03
I20260808 02:16:55.986686 5172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:56.001444 5172 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:56.002925 5172 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:56.010722 5172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93: Generated new TSK 0
I20260808 02:16:56.011605 5172 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:16:56.044919 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46717
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:39831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:56.173488 5187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:56.173908 5187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:56.174003 5187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:56.180081 5187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:56.180251 5187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:56.180465 5187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:16:56.189409 5187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:39831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:56.190913 5187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:56.192406 5187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:56.205071 5194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:56.205071 5193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:56.205070 5197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:56.206015 5187 server_base.cc:1034] running on GCE node
I20260808 02:16:56.206419 5187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:56.207104 5187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:56.208277 5187 hybrid_clock.cc:648] HybridClock initialized: now 1786155416208259 us; error 50 us; skew 500 ppm
Aug 08 02:16:56 dist-test-slave-9gcw krb5kdc[5071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:56.211269 5187 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:16:56.212280 5187 webserver.cc:460] Webserver started at http://127.18.184.129:36751/ using document root <none> and password file <none>
I20260808 02:16:56.212888 5187 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:56.212949 5187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:56.213249 5187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:56.215390 5187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "9ada13a071fd438498b06de9afb02731"
format_stamp: "Formatted at 2026-08-08 02:16:56 on dist-test-slave-9gcw"
server_key: "6ee6f347c8b328b00ad2694c0cdb00bb"
server_key_iv: "d1c2652dfdff6f44885357b938f2a226"
server_key_version: "encryptionkey@0"
I20260808 02:16:56.216012 5187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "9ada13a071fd438498b06de9afb02731"
format_stamp: "Formatted at 2026-08-08 02:16:56 on dist-test-slave-9gcw"
server_key: "6ee6f347c8b328b00ad2694c0cdb00bb"
server_key_iv: "d1c2652dfdff6f44885357b938f2a226"
server_key_version: "encryptionkey@0"
I20260808 02:16:56.220276 5187 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:16:56.222913 5205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:56.224227 5187 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:16:56.224376 5187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "9ada13a071fd438498b06de9afb02731"
format_stamp: "Formatted at 2026-08-08 02:16:56 on dist-test-slave-9gcw"
server_key: "6ee6f347c8b328b00ad2694c0cdb00bb"
server_key_iv: "d1c2652dfdff6f44885357b938f2a226"
server_key_version: "encryptionkey@0"
I20260808 02:16:56.224496 5187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:56.242643 5187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:56.245743 5187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:56.245963 5187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:56.246609 5187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:56.247467 5187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:56.247536 5187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:56.247608 5187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:56.247660 5187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:56.256897 5187 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:39971
I20260808 02:16:56.256927 5334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:39971 every 8 connection(s)
I20260808 02:16:56.257834 5187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:16:56.263466 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 5187
I20260808 02:16:56.263635 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:16:56.263967 19170 external_mini_cluster.cc:1428] Setting key 44ccd96de299029a20f8436626f12a91
Aug 08 02:16:56 dist-test-slave-9gcw krb5kdc[5071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:16:56.274192 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.259731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:58155 (local address 127.18.184.190:46717)
0808 02:16:56.259885 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:56.259889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:56.260691 (+ 802us) server_negotiation.cc:407] Connection header received
0808 02:16:56.262203 (+ 1512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:56.262206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:56.262272 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:56.262384 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:56.263652 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.264329 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.266110 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.266367 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.266456 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.269634 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:56.269640 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:56.269646 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:56.269683 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:56.271668 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:56.272273 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:56.272280 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:56.272285 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:56.272390 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:56.272776 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:56.272781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:56.272785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:56.273086 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:56.273232 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:56.273800 (+ 568us) server_negotiation.cc:299] Negotiation successful
0808 02:16:56.273984 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:16:56.274979 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.260072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:46717 (local address 127.18.184.129:58155)
0808 02:16:56.260529 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:56.260574 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:56.261977 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:56.262566 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:56.262574 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:56.262980 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:56.263489 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:56.263503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.264512 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.264516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:56.265947 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:56.265963 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.266539 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.266546 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:56.266810 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.267540 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:56.267567 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:56.269437 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:56.271817 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:56.271823 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:56.271837 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:56.272143 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:56.272501 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:56.272503 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:56.272505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:56.272662 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:56.273258 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:56.273263 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:56.273528 (+ 265us) client_negotiation.cc:769] Sending connection context
0808 02:16:56.273686 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 02:16:56.274020 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:16:56.276523 5336 heartbeater.cc:347] Connected to a master server at 127.18.184.190:46717
I20260808 02:16:56.276804 5336 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:56.277439 5336 heartbeater.cc:511] Master 127.18.184.190:46717 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
I20260808 02:16:56.279186 5121 ts_manager.cc:194] Registered new tserver with Master: 9ada13a071fd438498b06de9afb02731 (127.18.184.129:39971)
I20260808 02:16:56.280808 5121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:58155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:16:56.296979 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46717
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:39831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:56.413687 5342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:56.413947 5342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:56.413991 5342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:56.418232 5342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:56.418334 5342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:56.418437 5342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:16:56.423957 5342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:39831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:56.425107 5342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:56.426257 5342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:56.436697 5351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:56.436856 5342 server_base.cc:1034] running on GCE node
W20260808 02:16:56.436707 5347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:56.437037 5348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:56.437539 5342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:56.438151 5342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:56.439379 5342 hybrid_clock.cc:648] HybridClock initialized: now 1786155416439362 us; error 26 us; skew 500 ppm
Aug 08 02:16:56 dist-test-slave-9gcw krb5kdc[5071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:56.442703 5342 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:16:56.443673 5342 webserver.cc:460] Webserver started at http://127.18.184.130:44787/ using document root <none> and password file <none>
I20260808 02:16:56.444279 5342 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:56.444350 5342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:56.444621 5342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:56.446832 5342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "51d7d9dd5be04809a4d90190b18c6f07"
format_stamp: "Formatted at 2026-08-08 02:16:56 on dist-test-slave-9gcw"
server_key: "0ff7d068a5a246f84265aa4dfa90a066"
server_key_iv: "b2e6c84bea98081eceea93a554b0a415"
server_key_version: "encryptionkey@0"
I20260808 02:16:56.447448 5342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "51d7d9dd5be04809a4d90190b18c6f07"
format_stamp: "Formatted at 2026-08-08 02:16:56 on dist-test-slave-9gcw"
server_key: "0ff7d068a5a246f84265aa4dfa90a066"
server_key_iv: "b2e6c84bea98081eceea93a554b0a415"
server_key_version: "encryptionkey@0"
I20260808 02:16:56.451431 5342 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 02:16:56.454113 5358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:56.455693 5342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:56.455874 5342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "51d7d9dd5be04809a4d90190b18c6f07"
format_stamp: "Formatted at 2026-08-08 02:16:56 on dist-test-slave-9gcw"
server_key: "0ff7d068a5a246f84265aa4dfa90a066"
server_key_iv: "b2e6c84bea98081eceea93a554b0a415"
server_key_version: "encryptionkey@0"
I20260808 02:16:56.456020 5342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:56.490062 5342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:56.494093 5342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:56.494351 5342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:56.495035 5342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:56.495986 5342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:56.496064 5342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:56.496141 5342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:56.496194 5342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:56.506152 5342 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:38907
I20260808 02:16:56.506181 5492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:38907 every 8 connection(s)
I20260808 02:16:56.507138 5342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:16:56.515300 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 5342
I20260808 02:16:56.515481 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:16:56.515793 19170 external_mini_cluster.cc:1428] Setting key 25ddfa428f886cd2684f8067d0ba8a4c
Aug 08 02:16:56 dist-test-slave-9gcw krb5kdc[5071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:16:56.522585 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.509219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:46231 (local address 127.18.184.190:46717)
0808 02:16:56.509397 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:56.509402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:56.510209 (+ 807us) server_negotiation.cc:407] Connection header received
0808 02:16:56.511919 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:56.511922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:56.511970 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:56.512045 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:56.513516 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.514057 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.515433 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.515722 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.515804 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.518805 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:56.518809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:56.518811 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:56.518840 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:56.520375 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:56.520948 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:56.520957 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:56.520959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:56.521036 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:56.521436 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:56.521439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:56.521441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:56.521631 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:56.521742 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:56.522278 (+ 536us) server_negotiation.cc:299] Negotiation successful
0808 02:16:56.522414 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:16:56.523454 5495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.509525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:46717 (local address 127.18.184.130:46231)
0808 02:16:56.510023 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:56.510079 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:56.511622 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:56.512198 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:56.512205 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:56.512735 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:56.513364 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:56.513388 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.514194 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.514197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:56.515268 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:56.515280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.515924 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.515929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:56.516129 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.516811 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:56.516828 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:56.518633 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:56.520505 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:56.520511 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:56.520528 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:56.520829 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:56.521134 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:56.521137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:56.521140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:56.521320 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:56.521743 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:56.521753 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:56.522029 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 02:16:56.522220 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 02:16:56.522570 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":123,"threads_started":1}
I20260808 02:16:56.524709 5493 heartbeater.cc:347] Connected to a master server at 127.18.184.190:46717
I20260808 02:16:56.524999 5493 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:56.525663 5493 heartbeater.cc:511] Master 127.18.184.190:46717 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:16:56.526988 5121 ts_manager.cc:194] Registered new tserver with Master: 51d7d9dd5be04809a4d90190b18c6f07 (127.18.184.130:38907)
I20260808 02:16:56.527678 5121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:46231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:16:56.545774 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46717
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:39831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:56.664178 5498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:56.664449 5498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:56.664497 5498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:56.668612 5498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:56.668678 5498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:56.668737 5498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:16:56.674263 5498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:39831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:56.675336 5498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:56.676403 5498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:56.687023 5504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:56.687177 5503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:56.687024 5506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:56.687093 5498 server_base.cc:1034] running on GCE node
I20260808 02:16:56.687678 5498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:56.688293 5498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:56.689455 5498 hybrid_clock.cc:648] HybridClock initialized: now 1786155416689444 us; error 26 us; skew 500 ppm
Aug 08 02:16:56 dist-test-slave-9gcw krb5kdc[5071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:56.692394 5498 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:16:56.693449 5498 webserver.cc:460] Webserver started at http://127.18.184.131:43739/ using document root <none> and password file <none>
I20260808 02:16:56.694063 5498 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:56.694125 5498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:56.694375 5498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:56.696516 5498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "1e3856b95d9f4ee9af12b599a7510e36"
format_stamp: "Formatted at 2026-08-08 02:16:56 on dist-test-slave-9gcw"
server_key: "2c4bc2e1adeecec6494f05ca763fc45c"
server_key_iv: "dc345ab8d39c5502ef675350bc242f91"
server_key_version: "encryptionkey@0"
I20260808 02:16:56.697104 5498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "1e3856b95d9f4ee9af12b599a7510e36"
format_stamp: "Formatted at 2026-08-08 02:16:56 on dist-test-slave-9gcw"
server_key: "2c4bc2e1adeecec6494f05ca763fc45c"
server_key_iv: "dc345ab8d39c5502ef675350bc242f91"
server_key_version: "encryptionkey@0"
I20260808 02:16:56.701033 5498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:16:56.703673 5513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:56.704970 5498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:56.705117 5498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "1e3856b95d9f4ee9af12b599a7510e36"
format_stamp: "Formatted at 2026-08-08 02:16:56 on dist-test-slave-9gcw"
server_key: "2c4bc2e1adeecec6494f05ca763fc45c"
server_key_iv: "dc345ab8d39c5502ef675350bc242f91"
server_key_version: "encryptionkey@0"
I20260808 02:16:56.705260 5498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:56.732120 5498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:56.735385 5498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:56.735605 5498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:56.736239 5498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:56.737120 5498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:56.737223 5498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:56.737299 5498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:56.737349 5498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:56.746996 5498 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:36259
I20260808 02:16:56.747032 5637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:36259 every 8 connection(s)
I20260808 02:16:56.747982 5498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:16:56.754027 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 5498
I20260808 02:16:56.754186 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:16:56.754531 19170 external_mini_cluster.cc:1428] Setting key 0661e8cb87c4e4ec63652fe05c15ee76
Aug 08 02:16:56 dist-test-slave-9gcw krb5kdc[5071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:16:56.762933 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.750053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:60021 (local address 127.18.184.190:46717)
0808 02:16:56.750202 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:56.750207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:56.751013 (+ 806us) server_negotiation.cc:407] Connection header received
0808 02:16:56.752455 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:56.752458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:56.752520 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:56.752635 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:56.753948 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.754639 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.756166 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.756421 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.756534 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.759333 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:56.759336 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:56.759338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:56.759364 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:56.760848 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:56.761496 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:56.761498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:56.761500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:56.761581 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:56.761915 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:56.761918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:56.761919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:56.762107 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:56.762182 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:56.762695 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 02:16:56.762803 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:16:56.763893 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.750374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:46717 (local address 127.18.184.131:60021)
0808 02:16:56.750845 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:56.750897 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:56.752210 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:56.752822 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:56.752829 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:56.753280 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:56.753788 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:56.753804 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.754768 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.754773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:56.756031 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:56.756041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.756594 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.756599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:56.756816 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.757537 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:56.757554 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:56.759156 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:56.760974 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:56.760979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:56.760994 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:56.761403 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:56.761666 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:56.761668 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:56.761670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:56.761823 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:56.762198 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:56.762203 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:56.762476 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 02:16:56.762634 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 02:16:56.762928 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":113,"threads_started":1}
I20260808 02:16:56.765476 5638 heartbeater.cc:347] Connected to a master server at 127.18.184.190:46717
I20260808 02:16:56.765770 5638 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:56.766355 5638 heartbeater.cc:511] Master 127.18.184.190:46717 requested a full tablet report, sending...
I20260808 02:16:56.767674 5121 ts_manager.cc:194] Registered new tserver with Master: 1e3856b95d9f4ee9af12b599a7510e36 (127.18.184.131:36259)
I20260808 02:16:56.768486 5121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:60021
I20260808 02:16:56.768887 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:16:56.779484 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.771144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41544 (local address 127.18.184.190:46717)
0808 02:16:56.771276 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:56.771281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:56.771553 (+ 272us) server_negotiation.cc:407] Connection header received
0808 02:16:56.771606 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:56.771608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:56.771653 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:56.771809 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:56.772619 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.773356 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.774219 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.774448 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.774543 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.776145 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:56.776148 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:56.776151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:56.776184 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:56.777751 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:56.778223 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:56.778226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:56.778227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:56.778311 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:56.778616 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:56.778622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:56.778624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:56.778923 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:56.779035 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:56.779250 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 02:16:56.779354 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:16:56.783196 5121 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:16:56.786441 5121 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:16:56.802385 5653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.796910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59584 (local address 127.18.184.131:36259)
0808 02:16:56.797719 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:56.797732 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:56.797766 (+ 34us) server_negotiation.cc:407] Connection header received
0808 02:16:56.797840 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:56.797844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:56.797984 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:56.798142 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:56.798382 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.799286 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.800689 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.801490 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.801626 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.801767 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:56.802126 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 02:16:56.802234 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":189,"threads_started":1}
I20260808 02:16:56.802376 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.796994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:39971 (local address 127.0.0.1:44198)
0808 02:16:56.797369 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:56.797384 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:56.797472 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:56.798304 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:56.798310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:56.798331 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:56.798427 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:56.798432 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.799726 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.799729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:56.800863 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:56.800872 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.801951 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.801954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:56.802039 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.802063 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:16:56.802142 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 02:16:56.802197 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":33,"thread_start_us":103,"threads_started":1}
I20260808 02:16:56.802376 5649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.795813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:38907 (local address 127.0.0.1:53068)
0808 02:16:56.796252 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:56.796311 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:56.796460 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:56.797294 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:56.797301 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:56.797325 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:56.797456 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:56.797461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.798896 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.798900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:56.800379 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:56.800390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.801701 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.801705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:56.801819 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.801861 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 02:16:56.801976 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 02:16:56.802136 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":62,"thread_start_us":102,"threads_started":1}
I20260808 02:16:56.802528 5654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.797479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44198 (local address 127.18.184.129:39971)
0808 02:16:56.797785 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:56.797792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:56.797816 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:16:56.797891 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:56.797896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:56.798069 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:56.798200 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:56.798555 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.799590 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.800996 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.801813 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.801952 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.802065 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:56.802280 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 02:16:56.802365 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":109,"threads_started":1}
I20260808 02:16:56.802805 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.795947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53068 (local address 127.18.184.130:38907)
0808 02:16:56.796518 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:56.796533 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:56.796560 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:16:56.796640 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:56.796645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:56.796866 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:56.797030 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:56.797583 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.798741 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.800512 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.801344 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.801475 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.801609 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:56.802547 (+ 938us) server_negotiation.cc:299] Negotiation successful
0808 02:16:56.802654 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":172,"threads_started":1}
I20260808 02:16:56.802842 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.796600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:36259 (local address 127.0.0.1:59584)
0808 02:16:56.796934 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:56.796947 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:56.797062 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:56.798131 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:56.798134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:56.798152 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:56.798267 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:56.798282 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.799470 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.799474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:56.800558 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:56.800571 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.801825 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.801828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:56.801908 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.801935 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:16:56.801997 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 02:16:56.802173 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":61,"thread_start_us":104,"threads_started":1}
I20260808 02:16:56.806157 5259 tablet_service.cc:1467] Processing CreateTablet for tablet 60b30db5b4c74af2b56c16b3c56ceddf (DEFAULT_TABLE table=test-table [id=b5e8c062360945249479ccde7b12db95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:56.806192 5412 tablet_service.cc:1467] Processing CreateTablet for tablet 60b30db5b4c74af2b56c16b3c56ceddf (DEFAULT_TABLE table=test-table [id=b5e8c062360945249479ccde7b12db95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:56.806191 5566 tablet_service.cc:1467] Processing CreateTablet for tablet 60b30db5b4c74af2b56c16b3c56ceddf (DEFAULT_TABLE table=test-table [id=b5e8c062360945249479ccde7b12db95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:56.807214 5259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60b30db5b4c74af2b56c16b3c56ceddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:56.807231 5412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60b30db5b4c74af2b56c16b3c56ceddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:56.807231 5566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60b30db5b4c74af2b56c16b3c56ceddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:56.814499 5655 tablet_bootstrap.cc:492] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731: Bootstrap starting.
I20260808 02:16:56.815547 5656 tablet_bootstrap.cc:492] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36: Bootstrap starting.
I20260808 02:16:56.815547 5657 tablet_bootstrap.cc:492] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07: Bootstrap starting.
I20260808 02:16:56.816960 5655 tablet_bootstrap.cc:654] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:56.817840 5656 tablet_bootstrap.cc:654] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:56.817840 5657 tablet_bootstrap.cc:654] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:56.817822 5655 log.cc:824] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:56.819322 5656 log.cc:824] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:56.819322 5657 log.cc:824] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:56.820297 5655 tablet_bootstrap.cc:492] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731: No bootstrap required, opened a new log
I20260808 02:16:56.820501 5655 ts_tablet_manager.cc:1397] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 02:16:56.821210 5656 tablet_bootstrap.cc:492] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36: No bootstrap required, opened a new log
I20260808 02:16:56.821401 5656 ts_tablet_manager.cc:1397] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:16:56.822435 5657 tablet_bootstrap.cc:492] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07: No bootstrap required, opened a new log
I20260808 02:16:56.822607 5657 ts_tablet_manager.cc:1397] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 02:16:56.824167 5655 raft_consensus.cc:348] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d7d9dd5be04809a4d90190b18c6f07" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38907 } } peers { permanent_uuid: "1e3856b95d9f4ee9af12b599a7510e36" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36259 } } peers { permanent_uuid: "9ada13a071fd438498b06de9afb02731" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 39971 } }
I20260808 02:16:56.824429 5655 raft_consensus.cc:374] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:56.824503 5655 raft_consensus.cc:729] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ada13a071fd438498b06de9afb02731, State: Initialized, Role: FOLLOWER
I20260808 02:16:56.824959 5655 consensus_queue.cc:260] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d7d9dd5be04809a4d90190b18c6f07" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38907 } } peers { permanent_uuid: "1e3856b95d9f4ee9af12b599a7510e36" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36259 } } peers { permanent_uuid: "9ada13a071fd438498b06de9afb02731" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 39971 } }
I20260808 02:16:56.825302 5656 raft_consensus.cc:348] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d7d9dd5be04809a4d90190b18c6f07" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38907 } } peers { permanent_uuid: "1e3856b95d9f4ee9af12b599a7510e36" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36259 } } peers { permanent_uuid: "9ada13a071fd438498b06de9afb02731" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 39971 } }
I20260808 02:16:56.825548 5656 raft_consensus.cc:374] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:56.825593 5656 raft_consensus.cc:729] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e3856b95d9f4ee9af12b599a7510e36, State: Initialized, Role: FOLLOWER
I20260808 02:16:56.825990 5655 ts_tablet_manager.cc:1428] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 02:16:56.826117 5656 consensus_queue.cc:260] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d7d9dd5be04809a4d90190b18c6f07" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38907 } } peers { permanent_uuid: "1e3856b95d9f4ee9af12b599a7510e36" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36259 } } peers { permanent_uuid: "9ada13a071fd438498b06de9afb02731" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 39971 } }
I20260808 02:16:56.826349 5336 heartbeater.cc:503] Master 127.18.184.190:46717 was elected leader, sending a full tablet report...
I20260808 02:16:56.826426 5657 raft_consensus.cc:348] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d7d9dd5be04809a4d90190b18c6f07" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38907 } } peers { permanent_uuid: "1e3856b95d9f4ee9af12b599a7510e36" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36259 } } peers { permanent_uuid: "9ada13a071fd438498b06de9afb02731" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 39971 } }
I20260808 02:16:56.826706 5657 raft_consensus.cc:374] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:56.826782 5657 raft_consensus.cc:729] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51d7d9dd5be04809a4d90190b18c6f07, State: Initialized, Role: FOLLOWER
I20260808 02:16:56.826901 5638 heartbeater.cc:503] Master 127.18.184.190:46717 was elected leader, sending a full tablet report...
I20260808 02:16:56.827160 5656 ts_tablet_manager.cc:1428] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:16:56.827178 5657 consensus_queue.cc:260] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d7d9dd5be04809a4d90190b18c6f07" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38907 } } peers { permanent_uuid: "1e3856b95d9f4ee9af12b599a7510e36" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36259 } } peers { permanent_uuid: "9ada13a071fd438498b06de9afb02731" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 39971 } }
I20260808 02:16:56.828145 5657 ts_tablet_manager.cc:1428] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 02:16:56.828320 5493 heartbeater.cc:503] Master 127.18.184.190:46717 was elected leader, sending a full tablet report...
W20260808 02:16:56.999122 5639 tablet.cc:2369] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:16:57.008467 5494 tablet.cc:2369] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:16:57.009400 5338 tablet.cc:2369] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:16:57.046810 5663 raft_consensus.cc:482] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:57.047062 5663 raft_consensus.cc:504] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d7d9dd5be04809a4d90190b18c6f07" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38907 } } peers { permanent_uuid: "1e3856b95d9f4ee9af12b599a7510e36" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36259 } } peers { permanent_uuid: "9ada13a071fd438498b06de9afb02731" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 39971 } }
I20260808 02:16:57.048338 5663 leader_election.cc:288] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51d7d9dd5be04809a4d90190b18c6f07 (127.18.184.130:38907), 1e3856b95d9f4ee9af12b599a7510e36 (127.18.184.131:36259)
I20260808 02:16:57.053010 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:57.048878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:52007 (local address 127.18.184.130:38907)
0808 02:16:57.049102 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:57.049108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:57.049325 (+ 217us) server_negotiation.cc:407] Connection header received
0808 02:16:57.049733 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:57.049736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:57.049787 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:57.049903 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:57.050252 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.050779 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.051988 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.052398 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.052544 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:57.052633 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:57.052773 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 02:16:57.052848 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260808 02:16:57.053079 5653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:57.049408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:44669 (local address 127.18.184.131:36259)
0808 02:16:57.049582 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:57.049585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:57.049729 (+ 144us) server_negotiation.cc:407] Connection header received
0808 02:16:57.049979 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:57.049984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:57.050036 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:57.050140 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:57.050454 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.050952 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.052058 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.052466 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.052601 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:57.052655 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:57.052875 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 02:16:57.052950 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:16:57.053100 5668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:57.049244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:36259 (local address 127.18.184.129:44669)
0808 02:16:57.049622 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:57.049633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:57.049771 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:57.050146 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:57.050149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:57.050196 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:57.050326 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:57.050332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.051073 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.051078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:57.051924 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:57.051934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.052602 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.052605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:57.052714 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:57.052738 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:16:57.052857 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 02:16:57.052928 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":101,"threads_started":1}
I20260808 02:16:57.053130 5667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:57.048688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:38907 (local address 127.18.184.129:52007)
0808 02:16:57.049194 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:57.049211 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:57.049388 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:57.049917 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:57.049921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:57.049940 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:57.050116 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:57.050123 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.050907 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.050914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:57.051837 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:57.051847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.052534 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.052539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:57.052637 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:57.052673 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:16:57.052756 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 02:16:57.052835 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":10,"thread_start_us":150,"threads_started":1}
I20260808 02:16:57.053836 5588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60b30db5b4c74af2b56c16b3c56ceddf" candidate_uuid: "9ada13a071fd438498b06de9afb02731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e3856b95d9f4ee9af12b599a7510e36" is_pre_election: true
I20260808 02:16:57.053895 5436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60b30db5b4c74af2b56c16b3c56ceddf" candidate_uuid: "9ada13a071fd438498b06de9afb02731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51d7d9dd5be04809a4d90190b18c6f07" is_pre_election: true
I20260808 02:16:57.054199 5588 raft_consensus.cc:2454] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ada13a071fd438498b06de9afb02731 in term 0.
I20260808 02:16:57.054208 5436 raft_consensus.cc:2454] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ada13a071fd438498b06de9afb02731 in term 0.
I20260808 02:16:57.054734 5207 leader_election.cc:302] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51d7d9dd5be04809a4d90190b18c6f07, 9ada13a071fd438498b06de9afb02731; no voters:
I20260808 02:16:57.055071 5663 raft_consensus.cc:2781] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:16:57.055186 5663 raft_consensus.cc:482] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:16:57.055240 5663 raft_consensus.cc:3037] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:57.056437 5663 raft_consensus.cc:504] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d7d9dd5be04809a4d90190b18c6f07" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38907 } } peers { permanent_uuid: "1e3856b95d9f4ee9af12b599a7510e36" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36259 } } peers { permanent_uuid: "9ada13a071fd438498b06de9afb02731" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 39971 } }
I20260808 02:16:57.056849 5663 leader_election.cc:288] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [CANDIDATE]: Term 1 election: Requested vote from peers 51d7d9dd5be04809a4d90190b18c6f07 (127.18.184.130:38907), 1e3856b95d9f4ee9af12b599a7510e36 (127.18.184.131:36259)
I20260808 02:16:57.057400 5588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60b30db5b4c74af2b56c16b3c56ceddf" candidate_uuid: "9ada13a071fd438498b06de9afb02731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e3856b95d9f4ee9af12b599a7510e36"
I20260808 02:16:57.057387 5436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60b30db5b4c74af2b56c16b3c56ceddf" candidate_uuid: "9ada13a071fd438498b06de9afb02731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51d7d9dd5be04809a4d90190b18c6f07"
I20260808 02:16:57.057550 5436 raft_consensus.cc:3037] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:57.057550 5588 raft_consensus.cc:3037] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:57.059034 5436 raft_consensus.cc:2454] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ada13a071fd438498b06de9afb02731 in term 1.
I20260808 02:16:57.059034 5588 raft_consensus.cc:2454] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ada13a071fd438498b06de9afb02731 in term 1.
I20260808 02:16:57.059405 5207 leader_election.cc:302] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e3856b95d9f4ee9af12b599a7510e36, 9ada13a071fd438498b06de9afb02731; no voters:
I20260808 02:16:57.059628 5663 raft_consensus.cc:2781] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:57.059953 5663 raft_consensus.cc:686] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [term 1 LEADER]: Becoming Leader. State: Replica: 9ada13a071fd438498b06de9afb02731, State: Running, Role: LEADER
I20260808 02:16:57.060381 5663 consensus_queue.cc:237] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d7d9dd5be04809a4d90190b18c6f07" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38907 } } peers { permanent_uuid: "1e3856b95d9f4ee9af12b599a7510e36" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36259 } } peers { permanent_uuid: "9ada13a071fd438498b06de9afb02731" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 39971 } }
I20260808 02:16:57.064005 5121 catalog_manager.cc:5495] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ada13a071fd438498b06de9afb02731 (127.18.184.129). New cstate: current_term: 1 leader_uuid: "9ada13a071fd438498b06de9afb02731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d7d9dd5be04809a4d90190b18c6f07" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e3856b95d9f4ee9af12b599a7510e36" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ada13a071fd438498b06de9afb02731" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 39971 } health_report { overall_health: HEALTHY } } }
I20260808 02:16:57.096024 5654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:57.092317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.18.184.129:39971)
0808 02:16:57.092464 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:57.092468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:57.092482 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:16:57.092586 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:57.092588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:57.092641 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:57.092728 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:57.092953 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.093527 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.094453 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.094666 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.094747 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:57.095081 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:57.095557 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:16:57.095654 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:57.095766 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 02:16:57.095828 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:16:57.102470 5588 raft_consensus.cc:1261] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36 [term 1 FOLLOWER]: Refusing update from remote peer 9ada13a071fd438498b06de9afb02731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:57.102470 5436 raft_consensus.cc:1261] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07 [term 1 FOLLOWER]: Refusing update from remote peer 9ada13a071fd438498b06de9afb02731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:57.103199 5663 consensus_queue.cc:1035] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51d7d9dd5be04809a4d90190b18c6f07" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:57.103401 5669 consensus_queue.cc:1035] T 60b30db5b4c74af2b56c16b3c56ceddf P 9ada13a071fd438498b06de9afb02731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e3856b95d9f4ee9af12b599a7510e36" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:57.111382 5673 mvcc.cc:205] Tried to move back new op lower bound from 7316092588443549696 to 7316092588282650624. Current Snapshot: MvccSnapshot[applied={T|T < 7316092588443549696}]
I20260808 02:16:57.112885 5674 mvcc.cc:205] Tried to move back new op lower bound from 7316092588443549696 to 7316092588282650624. Current Snapshot: MvccSnapshot[applied={T|T < 7316092588443549696}]
I20260808 02:16:57.121464 5120 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:57.121639 5120 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:57.124316 5120 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d4bc6d35b814be580e0dfc0f7d1fc93: Sending DeleteTablet for 3 replicas of tablet 60b30db5b4c74af2b56c16b3c56ceddf
I20260808 02:16:57.125252 5412 tablet_service.cc:1514] Processing DeleteTablet for tablet 60b30db5b4c74af2b56c16b3c56ceddf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:57 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:53068
I20260808 02:16:57.125519 5566 tablet_service.cc:1514] Processing DeleteTablet for tablet 60b30db5b4c74af2b56c16b3c56ceddf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:57 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:59584
I20260808 02:16:57.125838 5259 tablet_service.cc:1514] Processing DeleteTablet for tablet 60b30db5b4c74af2b56c16b3c56ceddf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:57 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:44198
I20260808 02:16:57.125977 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 5187
I20260808 02:16:57.126123 5684 tablet_replica.cc:330] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36: stopping tablet replica
I20260808 02:16:57.126504 5684 raft_consensus.cc:2229] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:57.126708 5683 tablet_replica.cc:330] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07: stopping tablet replica
I20260808 02:16:57.126973 5684 raft_consensus.cc:2258] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:57.127013 5683 raft_consensus.cc:2229] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:57.127274 5683 raft_consensus.cc:2258] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:57.129194 5683 ts_tablet_manager.cc:1905] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:57.129815 5684 ts_tablet_manager.cc:1905] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:57.132251 5683 ts_tablet_manager.cc:1918] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:57.132370 5683 log.cc:1196] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/60b30db5b4c74af2b56c16b3c56ceddf
I20260808 02:16:57.132669 5684 ts_tablet_manager.cc:1918] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:57.132710 5683 ts_tablet_manager.cc:1939] T 60b30db5b4c74af2b56c16b3c56ceddf P 51d7d9dd5be04809a4d90190b18c6f07: Deleting consensus metadata
I20260808 02:16:57.132777 5684 log.cc:1196] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/60b30db5b4c74af2b56c16b3c56ceddf
I20260808 02:16:57.133163 5684 ts_tablet_manager.cc:1939] T 60b30db5b4c74af2b56c16b3c56ceddf P 1e3856b95d9f4ee9af12b599a7510e36: Deleting consensus metadata
I20260808 02:16:57.134229 5104 catalog_manager.cc:4841] TS 51d7d9dd5be04809a4d90190b18c6f07 (127.18.184.130:38907): tablet 60b30db5b4c74af2b56c16b3c56ceddf (table test-table [id=b5e8c062360945249479ccde7b12db95]) successfully deleted
I20260808 02:16:57.134497 5104 catalog_manager.cc:4841] TS 1e3856b95d9f4ee9af12b599a7510e36 (127.18.184.131:36259): tablet 60b30db5b4c74af2b56c16b3c56ceddf (table test-table [id=b5e8c062360945249479ccde7b12db95]) successfully deleted
W20260808 02:16:57.136739 5104 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:39971 (error 108)
W20260808 02:16:57.138229 5652 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:16:57.137759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:39971 (local address 127.0.0.1:44208)
0808 02:16:57.137988 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:57.138075 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:39971: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":117}
W20260808 02:16:57.138429 5104 catalog_manager.cc:4584] TS 9ada13a071fd438498b06de9afb02731 (127.18.184.129:39971): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 60b30db5b4c74af2b56c16b3c56ceddf: Network error: Client connection negotiation failed: client connection to 127.18.184.129:39971: connect: Connection reset by peer (error 104)
I20260808 02:16:57.138492 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 5342
I20260808 02:16:57.146615 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 5498
I20260808 02:16:57.158280 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 5086
2026-08-08T02:16:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2709 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T02:16:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:57Z Disabled control of system clock
I20260808 02:16:57.192312 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:46011
--webserver_interface=127.18.184.190
--webserver_port=0
--builtin_ntp_servers=127.18.184.148:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:46011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:16:57.310730 5689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:57.311045 5689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:57.311085 5689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:57.315213 5689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:57.315299 5689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:57.315315 5689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:57.315332 5689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:57.315348 5689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:57.320477 5689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:46011
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:46011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:57.321684 5689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:57.323097 5689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:57.332420 5698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:57.332428 5695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:57.332584 5696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:57.332849 5689 server_base.cc:1034] running on GCE node
I20260808 02:16:57.333644 5689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:57.334913 5689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:57.336093 5689 hybrid_clock.cc:648] HybridClock initialized: now 1786155417336057 us; error 51 us; skew 500 ppm
I20260808 02:16:57.338245 5689 webserver.cc:460] Webserver started at http://127.18.184.190:41213/ using document root <none> and password file <none>
I20260808 02:16:57.338874 5689 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:57.338943 5689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:57.339178 5689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:57.341492 5689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "676ef8bc055c43b4a8a5d4e9ccc8cd77"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-9gcw"
server_key: "a3badb799f47f7e90b70408391dcf9a0"
server_key_iv: "583c349011585aa3492b8c27995397ba"
server_key_version: "encryptionkey@0"
I20260808 02:16:57.342119 5689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "676ef8bc055c43b4a8a5d4e9ccc8cd77"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-9gcw"
server_key: "a3badb799f47f7e90b70408391dcf9a0"
server_key_iv: "583c349011585aa3492b8c27995397ba"
server_key_version: "encryptionkey@0"
I20260808 02:16:57.346225 5689 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:16:57.348861 5705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:57.350219 5689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:16:57.350382 5689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "676ef8bc055c43b4a8a5d4e9ccc8cd77"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-9gcw"
server_key: "a3badb799f47f7e90b70408391dcf9a0"
server_key_iv: "583c349011585aa3492b8c27995397ba"
server_key_version: "encryptionkey@0"
I20260808 02:16:57.350510 5689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:57.368981 5689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:57.369808 5689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:57.370042 5689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:57.377697 5689 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:46011
I20260808 02:16:57.377712 5761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:46011 every 8 connection(s)
I20260808 02:16:57.378742 5689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:57.382253 5762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:57.388346 5762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77: Bootstrap starting.
I20260808 02:16:57.389060 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 5689
I20260808 02:16:57.389257 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:16:57.389567 19170 external_mini_cluster.cc:1428] Setting key 8990f153b56dddc3215a6aa9bbf6d38a
I20260808 02:16:57.391525 5762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:57.392355 5762 log.cc:824] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:57.394765 5762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77: No bootstrap required, opened a new log
I20260808 02:16:57.399505 5762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "676ef8bc055c43b4a8a5d4e9ccc8cd77" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46011 } }
I20260808 02:16:57.399803 5762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:57.399852 5762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676ef8bc055c43b4a8a5d4e9ccc8cd77, State: Initialized, Role: FOLLOWER
I20260808 02:16:57.400339 5765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:57.391063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42228 (local address 127.18.184.190:46011)
0808 02:16:57.391800 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:57.391815 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:57.391851 (+ 36us) server_negotiation.cc:407] Connection header received
0808 02:16:57.392554 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:57.392573 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:57.392977 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:57.393384 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:57.393821 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.395247 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.396173 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.396898 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.397066 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:57.397973 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:57.397982 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:57.398004 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:57.398652 (+ 648us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:57.398724 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:57.398751 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:57.398882 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:57.399025 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 02:16:57.399329 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":159,"threads_started":1}
I20260808 02:16:57.400334 5762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "676ef8bc055c43b4a8a5d4e9ccc8cd77" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46011 } }
I20260808 02:16:57.400502 5762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:57.400568 5762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:57.400673 5762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:57.401865 5762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "676ef8bc055c43b4a8a5d4e9ccc8cd77" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46011 } }
I20260808 02:16:57.402218 5762 leader_election.cc:302] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 676ef8bc055c43b4a8a5d4e9ccc8cd77; no voters:
I20260808 02:16:57.402583 5762 leader_election.cc:288] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:57.402719 5767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:57.403000 5767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [term 1 LEADER]: Becoming Leader. State: Replica: 676ef8bc055c43b4a8a5d4e9ccc8cd77, State: Running, Role: LEADER
I20260808 02:16:57.403366 5767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "676ef8bc055c43b4a8a5d4e9ccc8cd77" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46011 } }
I20260808 02:16:57.403882 5762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:57.404588 5767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 676ef8bc055c43b4a8a5d4e9ccc8cd77. Latest consensus state: current_term: 1 leader_uuid: "676ef8bc055c43b4a8a5d4e9ccc8cd77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "676ef8bc055c43b4a8a5d4e9ccc8cd77" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46011 } } }
I20260808 02:16:57.404726 5767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:57.405166 5775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:57.405311 5768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "676ef8bc055c43b4a8a5d4e9ccc8cd77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "676ef8bc055c43b4a8a5d4e9ccc8cd77" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 46011 } } }
I20260808 02:16:57.405486 5768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:57.408771 5775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:57.414515 5775 catalog_manager.cc:1347] Generated new cluster ID: cba7985d55a240a7a0d8a9f4049fd00a
I20260808 02:16:57.414594 5775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:57.428637 5775 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:57.429864 5775 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:57.434223 5775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77: Generated new TSK 0
I20260808 02:16:57.435058 5775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:16:57.442940 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46011
--builtin_ntp_servers=127.18.184.148:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:16:57.569577 5788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:57.569983 5788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:57.570071 5788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:57.574425 5788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:57.574508 5788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:57.574599 5788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:16:57.579545 5788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:57.580811 5788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:57.582340 5788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:57.592500 5794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:57.592492 5793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:57.592504 5796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:57.592921 5788 server_base.cc:1034] running on GCE node
I20260808 02:16:57.593403 5788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:57.594116 5788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:57.595314 5788 hybrid_clock.cc:648] HybridClock initialized: now 1786155417595283 us; error 56 us; skew 500 ppm
I20260808 02:16:57.597343 5788 webserver.cc:460] Webserver started at http://127.18.184.129:37185/ using document root <none> and password file <none>
I20260808 02:16:57.597951 5788 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:57.598047 5788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:57.598284 5788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:57.600518 5788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "56d942df246d4da8998a6be9b1379a8c"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-9gcw"
server_key: "c9c28b4ed7ab380e1fa1c84f8b667417"
server_key_iv: "56f06ae84f683ad64445aa094c089e7d"
server_key_version: "encryptionkey@0"
I20260808 02:16:57.601112 5788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "56d942df246d4da8998a6be9b1379a8c"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-9gcw"
server_key: "c9c28b4ed7ab380e1fa1c84f8b667417"
server_key_iv: "56f06ae84f683ad64445aa094c089e7d"
server_key_version: "encryptionkey@0"
I20260808 02:16:57.605374 5788 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:16:57.608004 5803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:57.609431 5788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:57.609556 5788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "56d942df246d4da8998a6be9b1379a8c"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-9gcw"
server_key: "c9c28b4ed7ab380e1fa1c84f8b667417"
server_key_iv: "56f06ae84f683ad64445aa094c089e7d"
server_key_version: "encryptionkey@0"
I20260808 02:16:57.609674 5788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:57.632184 5788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:57.632962 5788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:57.633244 5788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:57.633934 5788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:57.634963 5788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:57.635038 5788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:57.635110 5788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:57.635162 5788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:57.644667 5788 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:40945
I20260808 02:16:57.644750 5927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:40945 every 8 connection(s)
I20260808 02:16:57.645700 5788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:16:57.651397 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 5788
I20260808 02:16:57.651530 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:16:57.651872 19170 external_mini_cluster.cc:1428] Setting key e3e8a164fd811224358be265a14c5e3d
I20260808 02:16:57.654102 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46011
--builtin_ntp_servers=127.18.184.148:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:16:57.656839 5765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:57.647779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:48507 (local address 127.18.184.190:46011)
0808 02:16:57.647932 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:57.647936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:57.649031 (+ 1095us) server_negotiation.cc:407] Connection header received
0808 02:16:57.650622 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:57.650626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:57.650687 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:57.650803 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:57.652070 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.652767 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.654474 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.654690 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.654785 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:57.656234 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:57.656239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:57.656245 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:57.656262 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:57.656298 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:57.656311 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:57.656400 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:57.656564 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 02:16:57.656638 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:16:57.657711 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:57.648152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:46011 (local address 127.18.184.129:48507)
0808 02:16:57.648852 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:57.648896 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:57.650393 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:57.650984 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:57.650991 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:57.651125 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:57.651886 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:57.651910 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.652916 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.652919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:57.654352 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:57.654366 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.654873 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.654880 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:57.655125 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:57.655869 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:16:57.655874 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:57.655934 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:16:57.656029 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:57.656408 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:57.656416 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:57.656421 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:16:57.656527 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 02:16:57.656772 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":120,"threads_started":1}
I20260808 02:16:57.659029 5928 heartbeater.cc:347] Connected to a master server at 127.18.184.190:46011
I20260808 02:16:57.659309 5928 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:57.659997 5928 heartbeater.cc:511] Master 127.18.184.190:46011 requested a full tablet report, sending...
I20260808 02:16:57.662088 5722 ts_manager.cc:194] Registered new tserver with Master: 56d942df246d4da8998a6be9b1379a8c (127.18.184.129:40945)
I20260808 02:16:57.663381 5722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.184.129:48507
W20260808 02:16:57.772552 5931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:57.772841 5931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:57.772883 5931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:57.777107 5931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:57.777187 5931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:57.777300 5931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:16:57.782382 5931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:57.783658 5931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:57.785131 5931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:57.796146 5938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:57.796277 5931 server_base.cc:1034] running on GCE node
W20260808 02:16:57.796132 5940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:57.796132 5937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:57.796847 5931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:57.797676 5931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:57.798864 5931 hybrid_clock.cc:648] HybridClock initialized: now 1786155417798831 us; error 46 us; skew 500 ppm
I20260808 02:16:57.801004 5931 webserver.cc:460] Webserver started at http://127.18.184.130:41203/ using document root <none> and password file <none>
I20260808 02:16:57.801707 5931 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:57.801782 5931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:57.802028 5931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:57.804471 5931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "4a5a8b9516da4b96a7d80d126918b1a1"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-9gcw"
server_key: "f94243464ab02bb47b296e53f91a6a1c"
server_key_iv: "153615216f48f1d2b4a4fdf08600e4aa"
server_key_version: "encryptionkey@0"
I20260808 02:16:57.805385 5931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "4a5a8b9516da4b96a7d80d126918b1a1"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-9gcw"
server_key: "f94243464ab02bb47b296e53f91a6a1c"
server_key_iv: "153615216f48f1d2b4a4fdf08600e4aa"
server_key_version: "encryptionkey@0"
I20260808 02:16:57.809804 5931 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:16:57.812803 5946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:57.814356 5931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:57.814503 5931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "4a5a8b9516da4b96a7d80d126918b1a1"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-9gcw"
server_key: "f94243464ab02bb47b296e53f91a6a1c"
server_key_iv: "153615216f48f1d2b4a4fdf08600e4aa"
server_key_version: "encryptionkey@0"
I20260808 02:16:57.814621 5931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:57.833002 5931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:57.833840 5931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:57.834062 5931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:57.834702 5931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:57.835728 5931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:57.835812 5931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:57.835893 5931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:57.835947 5931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:57.845284 5931 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:41127
I20260808 02:16:57.845355 6069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:41127 every 8 connection(s)
I20260808 02:16:57.846259 5931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:16:57.852921 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 5931
I20260808 02:16:57.853076 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:16:57.853446 19170 external_mini_cluster.cc:1428] Setting key d368696c609a019e51034479d3304036
I20260808 02:16:57.855593 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46011
--builtin_ntp_servers=127.18.184.148:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:16:57.856557 5765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:57.848168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:41163 (local address 127.18.184.190:46011)
0808 02:16:57.848391 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:57.848396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:57.849514 (+ 1118us) server_negotiation.cc:407] Connection header received
0808 02:16:57.850953 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:57.850956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:57.851007 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:57.851086 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:57.852009 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.852540 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.854246 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.854483 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.854619 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:57.856024 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:57.856028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:57.856031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:57.856044 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:57.856072 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:57.856080 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:57.856160 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:57.856342 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 02:16:57.856407 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":78}
I20260808 02:16:57.857569 6073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:57.848641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:46011 (local address 127.18.184.130:41163)
0808 02:16:57.849353 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:57.849395 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:57.850721 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:57.851250 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:57.851257 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:57.851342 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:57.851873 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:57.851888 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.852667 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.852670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:57.854097 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:57.854109 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:57.854665 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:57.854669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:57.854894 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:57.855655 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:16:57.855656 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:57.855698 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:16:57.855850 (+ 152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:57.856163 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:57.856175 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:57.856183 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:16:57.856331 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 02:16:57.856650 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":135,"threads_started":1}
I20260808 02:16:57.858789 6071 heartbeater.cc:347] Connected to a master server at 127.18.184.190:46011
I20260808 02:16:57.859040 6071 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:57.859673 6071 heartbeater.cc:511] Master 127.18.184.190:46011 requested a full tablet report, sending...
I20260808 02:16:57.860888 5722 ts_manager.cc:194] Registered new tserver with Master: 4a5a8b9516da4b96a7d80d126918b1a1 (127.18.184.130:41127)
I20260808 02:16:57.861604 5722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.184.130:41163
W20260808 02:16:57.975489 6074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:57.975770 6074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:57.975811 6074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:57.979943 6074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:57.980014 6074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:57.980113 6074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:16:57.985231 6074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:46011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:57.986402 6074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:57.987846 6074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:57.997843 6079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:57.997859 6082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:57.997857 6080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:57.998090 6074 server_base.cc:1034] running on GCE node
I20260808 02:16:57.998596 6074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:57.999277 6074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:58.000470 6074 hybrid_clock.cc:648] HybridClock initialized: now 1786155418000453 us; error 28 us; skew 500 ppm
I20260808 02:16:58.002394 6074 webserver.cc:460] Webserver started at http://127.18.184.131:32921/ using document root <none> and password file <none>
I20260808 02:16:58.003000 6074 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:58.003095 6074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:58.003363 6074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:58.005766 6074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "41ecb26c5985444580ada50acc1aadb0"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-9gcw"
server_key: "8a80902f21605fcddb5e3790f3aa0f7d"
server_key_iv: "d59aec5e8e7ecb8d855da05f84b84e82"
server_key_version: "encryptionkey@0"
I20260808 02:16:58.006390 6074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "41ecb26c5985444580ada50acc1aadb0"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-9gcw"
server_key: "8a80902f21605fcddb5e3790f3aa0f7d"
server_key_iv: "d59aec5e8e7ecb8d855da05f84b84e82"
server_key_version: "encryptionkey@0"
I20260808 02:16:58.010190 6074 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:16:58.012794 6088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:58.014168 6074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:16:58.014309 6074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "41ecb26c5985444580ada50acc1aadb0"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-9gcw"
server_key: "8a80902f21605fcddb5e3790f3aa0f7d"
server_key_iv: "d59aec5e8e7ecb8d855da05f84b84e82"
server_key_version: "encryptionkey@0"
I20260808 02:16:58.014401 6074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:58.025704 6074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:58.026402 6074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:58.026605 6074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:58.027217 6074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:58.028170 6074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:58.028216 6074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:58.028290 6074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:58.028330 6074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:58.037751 6074 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:36607
I20260808 02:16:58.037779 6214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:36607 every 8 connection(s)
I20260808 02:16:58.038718 6074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:16:58.044212 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 6074
I20260808 02:16:58.044332 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:16:58.044651 19170 external_mini_cluster.cc:1428] Setting key a0aaba050b4a75e7f1741dbad9802557
I20260808 02:16:58.049088 5765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.040701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:45603 (local address 127.18.184.190:46011)
0808 02:16:58.040858 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:58.040862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:58.042038 (+ 1176us) server_negotiation.cc:407] Connection header received
0808 02:16:58.043511 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:58.043515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:58.043577 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:58.043686 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:58.044736 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.045450 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.047010 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.047265 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.047388 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.048538 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:58.048542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:58.048545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:58.048567 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:58.048587 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:58.048596 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:58.048692 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:58.048869 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 02:16:58.048935 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:16:58.050060 6217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.041115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:46011 (local address 127.18.184.131:45603)
0808 02:16:58.041881 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:58.041922 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:58.043285 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:58.043870 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:58.043877 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:58.043963 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:58.044585 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:58.044602 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.045617 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.045621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:58.046883 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:58.046893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.047433 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.047437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:58.047615 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.048259 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:16:58.048260 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:58.048305 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:16:58.048373 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:58.048686 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:58.048698 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:58.048706 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:16:58.048855 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 02:16:58.049199 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":125,"threads_started":1}
I20260808 02:16:58.051307 6215 heartbeater.cc:347] Connected to a master server at 127.18.184.190:46011
I20260808 02:16:58.051565 6215 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:58.052136 6215 heartbeater.cc:511] Master 127.18.184.190:46011 requested a full tablet report, sending...
I20260808 02:16:58.053362 5722 ts_manager.cc:194] Registered new tserver with Master: 41ecb26c5985444580ada50acc1aadb0 (127.18.184.131:36607)
I20260808 02:16:58.054078 5722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.184.131:45603
I20260808 02:16:58.058792 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:16:58.064973 5765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.061013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.18.184.190:46011)
0808 02:16:58.061238 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:58.061243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:58.061393 (+ 150us) server_negotiation.cc:407] Connection header received
0808 02:16:58.061549 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:58.061555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:58.061603 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:58.061690 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:58.061986 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.062507 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.063375 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.063594 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.063703 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.064491 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:58.064495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:58.064498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:58.064516 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:58.064534 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:58.064541 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:58.064620 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:58.064744 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 02:16:58.064813 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260808 02:16:58.067950 5722 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:16:58.071002 5722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:16:58.085791 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.080103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36764 (local address 127.18.184.131:36607)
0808 02:16:58.080612 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:58.080625 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:58.080648 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:16:58.080728 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:58.080733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:58.080932 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:58.081093 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:58.081452 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.082662 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.084322 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.085014 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.085146 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.085358 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:58.085503 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 02:16:58.085600 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":135,"threads_started":1}
I20260808 02:16:58.085779 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.079467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:36607 (local address 127.0.0.1:36764)
0808 02:16:58.079865 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:58.079924 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:58.080061 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:58.081100 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:58.081105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:58.081128 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:58.081314 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:58.081321 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.082773 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.082776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:58.084181 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:58.084193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.085167 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.085172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:58.085295 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.085334 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 02:16:58.085473 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 02:16:58.085600 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":105,"spinlock_wait_cycles":9472,"thread_start_us":126,"threads_started":1}
I20260808 02:16:58.086351 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.080296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:40945 (local address 127.0.0.1:48066)
0808 02:16:58.081397 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:58.081411 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:58.081544 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:58.081965 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:58.081968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:58.081987 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:58.082062 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:58.082068 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.083353 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.083359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:58.084617 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:58.084627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.085955 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.085959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:58.086060 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.086081 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 02:16:58.086144 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 02:16:58.086212 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1035,"mutex_wait_us":21,"thread_start_us":84,"threads_started":1}
I20260808 02:16:58.086505 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.079627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34632 (local address 127.18.184.130:41127)
0808 02:16:58.080102 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:58.080115 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:58.080394 (+ 279us) server_negotiation.cc:407] Connection header received
0808 02:16:58.080624 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:58.080631 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:58.080863 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:58.081009 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:58.081486 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.082485 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.084301 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.085047 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.085142 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.085371 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:58.085767 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 02:16:58.086249 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":143,"threads_started":1}
I20260808 02:16:58.086859 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.080488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48066 (local address 127.18.184.129:40945)
0808 02:16:58.080931 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:58.080944 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:58.081543 (+ 599us) server_negotiation.cc:407] Connection header received
0808 02:16:58.081646 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:58.081651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:58.081826 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:58.081966 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:58.082187 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.083237 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.084743 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.085574 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.085712 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.085878 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:58.086561 (+ 683us) server_negotiation.cc:299] Negotiation successful
0808 02:16:58.086678 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":150,"threads_started":1}
I20260808 02:16:58.087031 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.079467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:41127 (local address 127.0.0.1:34632)
0808 02:16:58.080289 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:58.080302 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:58.080471 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:58.081235 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:58.081238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:58.081251 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:58.081324 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:58.081329 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.082594 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.082599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:58.084141 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:58.084149 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.085485 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.085488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:58.085589 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.085626 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:16:58.086824 (+ 1198us) client_negotiation.cc:240] Negotiation successful
0808 02:16:58.086911 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":83,"thread_start_us":104,"threads_started":1}
I20260808 02:16:58.088711 6139 tablet_service.cc:1467] Processing CreateTablet for tablet b26700f84cd24e0eaeeef6544a400d3b (DEFAULT_TABLE table=test-table [id=e0fd5fedea1f40009de7197fbaead3b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:58.089787 6139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b26700f84cd24e0eaeeef6544a400d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:58.091957 5854 tablet_service.cc:1467] Processing CreateTablet for tablet b26700f84cd24e0eaeeef6544a400d3b (DEFAULT_TABLE table=test-table [id=e0fd5fedea1f40009de7197fbaead3b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:58.092489 5998 tablet_service.cc:1467] Processing CreateTablet for tablet b26700f84cd24e0eaeeef6544a400d3b (DEFAULT_TABLE table=test-table [id=e0fd5fedea1f40009de7197fbaead3b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:58.093040 5854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b26700f84cd24e0eaeeef6544a400d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:58.093499 5998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b26700f84cd24e0eaeeef6544a400d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:58.096752 6235 tablet_bootstrap.cc:492] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0: Bootstrap starting.
I20260808 02:16:58.098834 6235 tablet_bootstrap.cc:654] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:58.099606 6235 log.cc:824] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:58.101770 6235 tablet_bootstrap.cc:492] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0: No bootstrap required, opened a new log
I20260808 02:16:58.101783 6239 tablet_bootstrap.cc:492] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c: Bootstrap starting.
I20260808 02:16:58.101986 6235 ts_tablet_manager.cc:1397] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 02:16:58.103724 6239 tablet_bootstrap.cc:654] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:58.104538 6239 log.cc:824] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:58.104760 6237 tablet_bootstrap.cc:492] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1: Bootstrap starting.
I20260808 02:16:58.106077 6235 raft_consensus.cc:348] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5a8b9516da4b96a7d80d126918b1a1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41127 } } peers { permanent_uuid: "41ecb26c5985444580ada50acc1aadb0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36607 } } peers { permanent_uuid: "56d942df246d4da8998a6be9b1379a8c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40945 } }
I20260808 02:16:58.106323 6235 raft_consensus.cc:374] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:58.106375 6235 raft_consensus.cc:729] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ecb26c5985444580ada50acc1aadb0, State: Initialized, Role: FOLLOWER
I20260808 02:16:58.106835 6239 tablet_bootstrap.cc:492] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c: No bootstrap required, opened a new log
I20260808 02:16:58.106842 6235 consensus_queue.cc:260] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5a8b9516da4b96a7d80d126918b1a1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41127 } } peers { permanent_uuid: "41ecb26c5985444580ada50acc1aadb0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36607 } } peers { permanent_uuid: "56d942df246d4da8998a6be9b1379a8c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40945 } }
I20260808 02:16:58.107026 6239 ts_tablet_manager.cc:1397] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 02:16:58.107563 6215 heartbeater.cc:503] Master 127.18.184.190:46011 was elected leader, sending a full tablet report...
I20260808 02:16:58.107980 6235 ts_tablet_manager.cc:1428] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:16:58.108587 6237 tablet_bootstrap.cc:654] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:58.109495 6237 log.cc:824] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:58.111560 6237 tablet_bootstrap.cc:492] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1: No bootstrap required, opened a new log
I20260808 02:16:58.111358 6239 raft_consensus.cc:348] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5a8b9516da4b96a7d80d126918b1a1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41127 } } peers { permanent_uuid: "41ecb26c5985444580ada50acc1aadb0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36607 } } peers { permanent_uuid: "56d942df246d4da8998a6be9b1379a8c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40945 } }
I20260808 02:16:58.111625 6239 raft_consensus.cc:374] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:58.111692 6239 raft_consensus.cc:729] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56d942df246d4da8998a6be9b1379a8c, State: Initialized, Role: FOLLOWER
I20260808 02:16:58.111756 6237 ts_tablet_manager.cc:1397] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 02:16:58.112193 6239 consensus_queue.cc:260] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5a8b9516da4b96a7d80d126918b1a1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41127 } } peers { permanent_uuid: "41ecb26c5985444580ada50acc1aadb0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36607 } } peers { permanent_uuid: "56d942df246d4da8998a6be9b1379a8c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40945 } }
I20260808 02:16:58.113044 6239 ts_tablet_manager.cc:1428] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:16:58.113471 5928 heartbeater.cc:503] Master 127.18.184.190:46011 was elected leader, sending a full tablet report...
I20260808 02:16:58.115980 6237 raft_consensus.cc:348] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5a8b9516da4b96a7d80d126918b1a1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41127 } } peers { permanent_uuid: "41ecb26c5985444580ada50acc1aadb0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36607 } } peers { permanent_uuid: "56d942df246d4da8998a6be9b1379a8c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40945 } }
I20260808 02:16:58.116261 6237 raft_consensus.cc:374] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:58.116333 6237 raft_consensus.cc:729] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a5a8b9516da4b96a7d80d126918b1a1, State: Initialized, Role: FOLLOWER
I20260808 02:16:58.116809 6237 consensus_queue.cc:260] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5a8b9516da4b96a7d80d126918b1a1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41127 } } peers { permanent_uuid: "41ecb26c5985444580ada50acc1aadb0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36607 } } peers { permanent_uuid: "56d942df246d4da8998a6be9b1379a8c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40945 } }
I20260808 02:16:58.117487 6071 heartbeater.cc:503] Master 127.18.184.190:46011 was elected leader, sending a full tablet report...
I20260808 02:16:58.117787 6237 ts_tablet_manager.cc:1428] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:16:58.119638 6243 raft_consensus.cc:482] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:58.119813 6243 raft_consensus.cc:504] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5a8b9516da4b96a7d80d126918b1a1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41127 } } peers { permanent_uuid: "41ecb26c5985444580ada50acc1aadb0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36607 } } peers { permanent_uuid: "56d942df246d4da8998a6be9b1379a8c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40945 } }
I20260808 02:16:58.121766 6243 leader_election.cc:288] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a5a8b9516da4b96a7d80d126918b1a1 (127.18.184.130:41127), 41ecb26c5985444580ada50acc1aadb0 (127.18.184.131:36607)
I20260808 02:16:58.126291 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.121648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:59029 (local address 127.18.184.131:36607)
0808 02:16:58.121792 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:58.121796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:58.122118 (+ 322us) server_negotiation.cc:407] Connection header received
0808 02:16:58.122248 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:58.122251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:58.122312 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:58.122397 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:58.122700 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.123335 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.124774 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.125433 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.125567 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.125629 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:58.126065 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 02:16:58.126147 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:16:58.126291 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.121456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:45001 (local address 127.18.184.130:41127)
0808 02:16:58.121595 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:58.121602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:58.121825 (+ 223us) server_negotiation.cc:407] Connection header received
0808 02:16:58.121882 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:58.121887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:58.121933 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:58.122034 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:58.122342 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.123006 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.124766 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.125433 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.125567 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.125619 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:58.126065 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 02:16:58.126156 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 02:16:58.126364 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.121443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:41127 (local address 127.18.184.129:45001)
0808 02:16:58.121625 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:58.121641 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:58.121740 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:58.122072 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:58.122075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:58.122103 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:58.122232 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:58.122238 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.123236 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.123239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:58.124579 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:58.124591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.125602 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.125606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:58.125829 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.125904 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 02:16:58.126006 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 02:16:58.126151 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":99}
I20260808 02:16:58.126827 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.121476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:36607 (local address 127.18.184.129:59029)
0808 02:16:58.122029 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:58.122040 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:58.122162 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:58.122406 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:58.122409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:58.122425 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:58.122558 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:58.122563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.123451 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.123454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:58.124612 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:58.124622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.125602 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.125606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:58.125832 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.125904 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 02:16:58.126006 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 02:16:58.126151 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":157,"spinlock_wait_cycles":3328,"thread_start_us":124,"threads_started":1}
I20260808 02:16:58.127243 6019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b26700f84cd24e0eaeeef6544a400d3b" candidate_uuid: "56d942df246d4da8998a6be9b1379a8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a5a8b9516da4b96a7d80d126918b1a1" is_pre_election: true
I20260808 02:16:58.127610 6160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b26700f84cd24e0eaeeef6544a400d3b" candidate_uuid: "56d942df246d4da8998a6be9b1379a8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41ecb26c5985444580ada50acc1aadb0" is_pre_election: true
I20260808 02:16:58.127627 6019 raft_consensus.cc:2454] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56d942df246d4da8998a6be9b1379a8c in term 0.
I20260808 02:16:58.127887 6160 raft_consensus.cc:2454] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56d942df246d4da8998a6be9b1379a8c in term 0.
I20260808 02:16:58.128232 5804 leader_election.cc:302] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a5a8b9516da4b96a7d80d126918b1a1, 56d942df246d4da8998a6be9b1379a8c; no voters:
I20260808 02:16:58.128578 6243 raft_consensus.cc:2781] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:16:58.128734 6243 raft_consensus.cc:482] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:16:58.128805 6243 raft_consensus.cc:3037] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:58.130036 6243 raft_consensus.cc:504] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5a8b9516da4b96a7d80d126918b1a1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41127 } } peers { permanent_uuid: "41ecb26c5985444580ada50acc1aadb0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36607 } } peers { permanent_uuid: "56d942df246d4da8998a6be9b1379a8c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40945 } }
I20260808 02:16:58.130489 6243 leader_election.cc:288] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [CANDIDATE]: Term 1 election: Requested vote from peers 4a5a8b9516da4b96a7d80d126918b1a1 (127.18.184.130:41127), 41ecb26c5985444580ada50acc1aadb0 (127.18.184.131:36607)
I20260808 02:16:58.130873 6019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b26700f84cd24e0eaeeef6544a400d3b" candidate_uuid: "56d942df246d4da8998a6be9b1379a8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a5a8b9516da4b96a7d80d126918b1a1"
I20260808 02:16:58.131021 6019 raft_consensus.cc:3037] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:58.131031 6160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b26700f84cd24e0eaeeef6544a400d3b" candidate_uuid: "56d942df246d4da8998a6be9b1379a8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41ecb26c5985444580ada50acc1aadb0"
I20260808 02:16:58.131179 6160 raft_consensus.cc:3037] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:58.132247 6019 raft_consensus.cc:2454] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56d942df246d4da8998a6be9b1379a8c in term 1.
I20260808 02:16:58.132511 6160 raft_consensus.cc:2454] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56d942df246d4da8998a6be9b1379a8c in term 1.
I20260808 02:16:58.132591 5804 leader_election.cc:302] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a5a8b9516da4b96a7d80d126918b1a1, 56d942df246d4da8998a6be9b1379a8c; no voters:
I20260808 02:16:58.132843 6243 raft_consensus.cc:2781] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:58.133126 6243 raft_consensus.cc:686] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [term 1 LEADER]: Becoming Leader. State: Replica: 56d942df246d4da8998a6be9b1379a8c, State: Running, Role: LEADER
I20260808 02:16:58.133545 6243 consensus_queue.cc:237] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5a8b9516da4b96a7d80d126918b1a1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41127 } } peers { permanent_uuid: "41ecb26c5985444580ada50acc1aadb0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36607 } } peers { permanent_uuid: "56d942df246d4da8998a6be9b1379a8c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40945 } }
I20260808 02:16:58.137068 5722 catalog_manager.cc:5495] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 56d942df246d4da8998a6be9b1379a8c (127.18.184.129). New cstate: current_term: 1 leader_uuid: "56d942df246d4da8998a6be9b1379a8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5a8b9516da4b96a7d80d126918b1a1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ecb26c5985444580ada50acc1aadb0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56d942df246d4da8998a6be9b1379a8c" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40945 } health_report { overall_health: HEALTHY } } }
W20260808 02:16:58.146991 5929 tablet.cc:2369] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:16:58.150197 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.146398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48080 (local address 127.18.184.129:40945)
0808 02:16:58.146528 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:58.146533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:58.146548 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:16:58.146675 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:58.146678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:58.146737 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:58.146875 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:58.147204 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.147841 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.148709 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:58.148917 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:58.148992 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:58.149352 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:58.149758 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:16:58.149849 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:58.149969 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 02:16:58.150018 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:16:58.156468 6160 raft_consensus.cc:1261] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0 [term 1 FOLLOWER]: Refusing update from remote peer 56d942df246d4da8998a6be9b1379a8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:58.156468 6019 raft_consensus.cc:1261] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1 [term 1 FOLLOWER]: Refusing update from remote peer 56d942df246d4da8998a6be9b1379a8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:58.157205 6243 consensus_queue.cc:1035] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a5a8b9516da4b96a7d80d126918b1a1" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 41127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:58.157435 6247 consensus_queue.cc:1035] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "41ecb26c5985444580ada50acc1aadb0" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:58.164885 6250 mvcc.cc:205] Tried to move back new op lower bound from 7316092592761106432 to 7316092592678514688. Current Snapshot: MvccSnapshot[applied={T|T < 7316092592761106432}]
I20260808 02:16:58.166069 6251 mvcc.cc:205] Tried to move back new op lower bound from 7316092592761106432 to 7316092592678514688. Current Snapshot: MvccSnapshot[applied={T|T < 7316092592761106432}]
I20260808 02:16:58.174400 5721 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:58.174602 5721 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:58.177309 5721 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 676ef8bc055c43b4a8a5d4e9ccc8cd77: Sending DeleteTablet for 3 replicas of tablet b26700f84cd24e0eaeeef6544a400d3b
I20260808 02:16:58.178221 6139 tablet_service.cc:1514] Processing DeleteTablet for tablet b26700f84cd24e0eaeeef6544a400d3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:58 UTC) from {username='slave'} at 127.0.0.1:36764
I20260808 02:16:58.178333 5998 tablet_service.cc:1514] Processing DeleteTablet for tablet b26700f84cd24e0eaeeef6544a400d3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:58 UTC) from {username='slave'} at 127.0.0.1:34632
I20260808 02:16:58.179000 5854 tablet_service.cc:1514] Processing DeleteTablet for tablet b26700f84cd24e0eaeeef6544a400d3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:58 UTC) from {username='slave'} at 127.0.0.1:48066
I20260808 02:16:58.179255 6264 tablet_replica.cc:330] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1: stopping tablet replica
I20260808 02:16:58.179572 6265 tablet_replica.cc:330] T b26700f84cd24e0eaeeef6544a400d3b P 56d942df246d4da8998a6be9b1379a8c: stopping tablet replica
I20260808 02:16:58.179613 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 5788
I20260808 02:16:58.179622 6264 raft_consensus.cc:2229] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:58.179975 6264 raft_consensus.cc:2258] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:58.182118 6264 ts_tablet_manager.cc:1905] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:58.184379 6263 tablet_replica.cc:330] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0: stopping tablet replica
I20260808 02:16:58.184746 6263 raft_consensus.cc:2229] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:58.185081 6263 raft_consensus.cc:2258] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:58.185894 6264 ts_tablet_manager.cc:1918] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:58.185995 6264 log.cc:1196] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/b26700f84cd24e0eaeeef6544a400d3b
I20260808 02:16:58.186336 6264 ts_tablet_manager.cc:1939] T b26700f84cd24e0eaeeef6544a400d3b P 4a5a8b9516da4b96a7d80d126918b1a1: Deleting consensus metadata
I20260808 02:16:58.187288 6263 ts_tablet_manager.cc:1905] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:58.187592 5706 catalog_manager.cc:4841] TS 4a5a8b9516da4b96a7d80d126918b1a1 (127.18.184.130:41127): tablet b26700f84cd24e0eaeeef6544a400d3b (table test-table [id=e0fd5fedea1f40009de7197fbaead3b3]) successfully deleted
I20260808 02:16:58.191012 6263 ts_tablet_manager.cc:1918] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:58.191144 6263 log.cc:1196] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/b26700f84cd24e0eaeeef6544a400d3b
I20260808 02:16:58.191494 6263 ts_tablet_manager.cc:1939] T b26700f84cd24e0eaeeef6544a400d3b P 41ecb26c5985444580ada50acc1aadb0: Deleting consensus metadata
I20260808 02:16:58.193104 5707 catalog_manager.cc:4841] TS 41ecb26c5985444580ada50acc1aadb0 (127.18.184.131:36607): tablet b26700f84cd24e0eaeeef6544a400d3b (table test-table [id=e0fd5fedea1f40009de7197fbaead3b3]) successfully deleted
I20260808 02:16:58.194157 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 5931
W20260808 02:16:58.194178 5706 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:40945 (error 108)
I20260808 02:16:58.198026 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:58.197614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:40945 (local address 127.0.0.1:48096)
0808 02:16:58.197809 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:58.197908 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:40945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 02:16:58.198237 5706 catalog_manager.cc:4584] TS 56d942df246d4da8998a6be9b1379a8c (127.18.184.129:40945): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b26700f84cd24e0eaeeef6544a400d3b: Network error: Client connection negotiation failed: client connection to 127.18.184.129:40945: connect: Connection refused (error 111)
I20260808 02:16:58.202384 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 6074
I20260808 02:16:58.210552 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 5689
2026-08-08T02:16:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1052 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:58 dist-test-slave-9gcw krb5kdc[6269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:58 dist-test-slave-9gcw krb5kdc[6269](info): set up 2 sockets
Aug 08 02:16:58 dist-test-slave-9gcw krb5kdc[6269](info): commencing operation
krb5kdc: starting...
W20260808 02:16:59.367112 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:16:59 dist-test-slave-9gcw krb5kdc[6269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:16:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:16:59.505470 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:39301
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:39301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:59.622650 6282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:59.623031 6282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:59.623116 6282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:59.627476 6282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:59.627592 6282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:59.627633 6282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:59.627672 6282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:59.627735 6282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:59.633469 6282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:39301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:39301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:59.634713 6282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:59.635850 6282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:59.645248 6289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:59.645365 6282 server_base.cc:1034] running on GCE node
W20260808 02:16:59.645248 6288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:59.645248 6291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:59.646158 6282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:59.647406 6282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:59.648582 6282 hybrid_clock.cc:648] HybridClock initialized: now 1786155419648568 us; error 30 us; skew 500 ppm
Aug 08 02:16:59 dist-test-slave-9gcw krb5kdc[6269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155419, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:59.659093 6282 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:16:59.660233 6282 webserver.cc:460] Webserver started at http://127.18.184.190:33919/ using document root <none> and password file <none>
I20260808 02:16:59.660877 6282 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:59.660979 6282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:59.661309 6282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:59.663590 6282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "f6a6d2b285674e89837e5c282c4b0643"
format_stamp: "Formatted at 2026-08-08 02:16:59 on dist-test-slave-9gcw"
server_key: "4158fbd42648787ab929343954db4177"
server_key_iv: "745fdd38561192bdab7388d5a9c52bd2"
server_key_version: "encryptionkey@0"
I20260808 02:16:59.664202 6282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "f6a6d2b285674e89837e5c282c4b0643"
format_stamp: "Formatted at 2026-08-08 02:16:59 on dist-test-slave-9gcw"
server_key: "4158fbd42648787ab929343954db4177"
server_key_iv: "745fdd38561192bdab7388d5a9c52bd2"
server_key_version: "encryptionkey@0"
I20260808 02:16:59.668851 6282 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:16:59.672049 6300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:59.673491 6282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:59.673645 6282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "f6a6d2b285674e89837e5c282c4b0643"
format_stamp: "Formatted at 2026-08-08 02:16:59 on dist-test-slave-9gcw"
server_key: "4158fbd42648787ab929343954db4177"
server_key_iv: "745fdd38561192bdab7388d5a9c52bd2"
server_key_version: "encryptionkey@0"
I20260808 02:16:59.673775 6282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:59.698351 6282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:59.701968 6282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:59.702214 6282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:59.710492 6282 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:39301
I20260808 02:16:59.710855 6357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:39301 every 8 connection(s)
I20260808 02:16:59.711601 6282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:59.713842 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 6282
I20260808 02:16:59.713994 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:16:59.714306 19170 external_mini_cluster.cc:1428] Setting key 6b72d1fe0c62525093031e137ef16b5d
I20260808 02:16:59.714920 6360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:59.722591 6360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643: Bootstrap starting.
Aug 08 02:16:59 dist-test-slave-9gcw krb5kdc[6269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:16:59.725090 6360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:59.726027 6360 log.cc:824] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:59.728287 6360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643: No bootstrap required, opened a new log
I20260808 02:16:59.730953 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:59.715881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.18.184.190:39301)
0808 02:16:59.716356 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:59.716377 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:59.716428 (+ 51us) server_negotiation.cc:407] Connection header received
0808 02:16:59.717384 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:59.717410 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:59.717900 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:59.718323 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:59.719208 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:59.720705 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:59.721909 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:59.722603 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:59.722777 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:59.725694 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:59.725701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:59.725719 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:59.725754 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:59.727722 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:59.728249 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:59.728255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:59.728261 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:59.728484 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:59.728798 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:59.728802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:59.728804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:59.729320 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:59.729540 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:59.729832 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 02:16:59.730176 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":130,"threads_started":1}
I20260808 02:16:59.732226 6360 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a6d2b285674e89837e5c282c4b0643" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39301 } }
I20260808 02:16:59.732460 6360 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:59.732533 6360 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a6d2b285674e89837e5c282c4b0643, State: Initialized, Role: FOLLOWER
I20260808 02:16:59.733008 6360 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a6d2b285674e89837e5c282c4b0643" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39301 } }
I20260808 02:16:59.733196 6360 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:59.733266 6360 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:59.733402 6360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:59.734426 6360 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a6d2b285674e89837e5c282c4b0643" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39301 } }
I20260808 02:16:59.734786 6360 leader_election.cc:302] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6a6d2b285674e89837e5c282c4b0643; no voters:
I20260808 02:16:59.735078 6360 leader_election.cc:288] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:59.735247 6366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:59.735507 6366 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [term 1 LEADER]: Becoming Leader. State: Replica: f6a6d2b285674e89837e5c282c4b0643, State: Running, Role: LEADER
I20260808 02:16:59.735836 6366 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a6d2b285674e89837e5c282c4b0643" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39301 } }
I20260808 02:16:59.736413 6360 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:59.737721 6367 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6a6d2b285674e89837e5c282c4b0643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a6d2b285674e89837e5c282c4b0643" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39301 } } }
I20260808 02:16:59.737769 6368 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6a6d2b285674e89837e5c282c4b0643. Latest consensus state: current_term: 1 leader_uuid: "f6a6d2b285674e89837e5c282c4b0643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a6d2b285674e89837e5c282c4b0643" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 39301 } } }
I20260808 02:16:59.737871 6367 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:59.737900 6368 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:59.738303 6375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:59.741776 6375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:59.748010 6375 catalog_manager.cc:1347] Generated new cluster ID: 8331d747a7bc48f9b83482dae7d96e9e
I20260808 02:16:59.748227 6375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:59.762053 6375 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:59.763130 6375 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:59.776750 6375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643: Generated new TSK 0
I20260808 02:16:59.777652 6375 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:16:59.812242 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:39301
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:59.952848 6388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:59.953127 6388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:59.953262 6388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:59.957633 6388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:59.957701 6388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:59.957762 6388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:16:59.963150 6388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:39301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:16:59.964241 6388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:59.965334 6388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:59.975255 6394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:59.975286 6397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:59.975246 6395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:59.975689 6388 server_base.cc:1034] running on GCE node
I20260808 02:16:59.976127 6388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:59.976840 6388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:59.978087 6388 hybrid_clock.cc:648] HybridClock initialized: now 1786155419978067 us; error 34 us; skew 500 ppm
Aug 08 02:16:59 dist-test-slave-9gcw krb5kdc[6269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155419, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:59.981458 6388 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:16:59.982477 6388 webserver.cc:460] Webserver started at http://127.18.184.129:42877/ using document root <none> and password file <none>
I20260808 02:16:59.983088 6388 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:59.983155 6388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:59.983383 6388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:59.985566 6388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "4b30008d5a8b4ed486c9d992a27813ce"
format_stamp: "Formatted at 2026-08-08 02:16:59 on dist-test-slave-9gcw"
server_key: "7d335eefc4ff11d64dc80743499b2e02"
server_key_iv: "a28542a9886b6e22e45eb707a96f56ff"
server_key_version: "encryptionkey@0"
I20260808 02:16:59.986155 6388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "4b30008d5a8b4ed486c9d992a27813ce"
format_stamp: "Formatted at 2026-08-08 02:16:59 on dist-test-slave-9gcw"
server_key: "7d335eefc4ff11d64dc80743499b2e02"
server_key_iv: "a28542a9886b6e22e45eb707a96f56ff"
server_key_version: "encryptionkey@0"
I20260808 02:16:59.990263 6388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:16:59.992911 6406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:59.994318 6388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:16:59.994468 6388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "4b30008d5a8b4ed486c9d992a27813ce"
format_stamp: "Formatted at 2026-08-08 02:16:59 on dist-test-slave-9gcw"
server_key: "7d335eefc4ff11d64dc80743499b2e02"
server_key_iv: "a28542a9886b6e22e45eb707a96f56ff"
server_key_version: "encryptionkey@0"
I20260808 02:16:59.994594 6388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:00.010838 6388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:00.014199 6388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:00.014436 6388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:00.015112 6388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:00.016022 6388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:00.016098 6388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:00.016172 6388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:00.016216 6388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:00.025916 6388 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:32999
I20260808 02:17:00.025934 6533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:32999 every 8 connection(s)
I20260808 02:17:00.026880 6388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:17:00.031808 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 6388
I20260808 02:17:00.031961 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:17:00.032310 19170 external_mini_cluster.cc:1428] Setting key 571974c5eed53bfc67e22d6963b10428
Aug 08 02:17:00 dist-test-slave-9gcw krb5kdc[6269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155419, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:17:00.045009 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.028840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:34303 (local address 127.18.184.190:39301)
0808 02:17:00.029002 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.029006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.030021 (+ 1015us) server_negotiation.cc:407] Connection header received
0808 02:17:00.031419 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.031422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.031488 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.031596 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:00.033366 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.034150 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.036182 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.036590 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.036740 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.040104 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:00.040108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:00.040110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:00.040139 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:00.042203 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:00.042958 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:00.042963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:00.042965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:00.043081 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:00.043577 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:00.043582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:00.043584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:00.043881 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:00.044004 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.044592 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.044766 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:17:00.045979 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.029344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39301 (local address 127.18.184.129:34303)
0808 02:17:00.029843 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:00.029890 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:00.031168 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:00.031861 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:00.031872 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:00.032450 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:00.033160 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.033214 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.034339 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.034343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.035983 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.036001 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.036800 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.036804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.036999 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.037887 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:00.037913 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:00.039866 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:00.042391 (+ 2525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:00.042398 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:00.042416 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:00.042799 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:00.043244 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:00.043247 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:00.043249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:00.043438 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:00.044009 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:00.044015 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:00.044302 (+ 287us) client_negotiation.cc:769] Sending connection context
0808 02:17:00.044473 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 02:17:00.044903 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":112,"threads_started":1}
I20260808 02:17:00.047559 6534 heartbeater.cc:347] Connected to a master server at 127.18.184.190:39301
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
I20260808 02:17:00.047852 6534 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:00.048447 6534 heartbeater.cc:511] Master 127.18.184.190:39301 requested a full tablet report, sending...
I20260808 02:17:00.050156 6312 ts_manager.cc:194] Registered new tserver with Master: 4b30008d5a8b4ed486c9d992a27813ce (127.18.184.129:32999)
I20260808 02:17:00.051329 6312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:34303
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:17:00.064867 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:39301
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:00.183369 6539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:00.183660 6539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:00.183707 6539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:00.188104 6539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:00.188180 6539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:00.188275 6539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:17:00.194522 6539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:39301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:00.195750 6539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:00.196875 6539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:00.207378 6547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:00.207367 6544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:00.207369 6545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:00.207734 6539 server_base.cc:1034] running on GCE node
I20260808 02:17:00.208235 6539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:00.208925 6539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:00.210124 6539 hybrid_clock.cc:648] HybridClock initialized: now 1786155420210120 us; error 53 us; skew 500 ppm
Aug 08 02:17:00 dist-test-slave-9gcw krb5kdc[6269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155420, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:00.213088 6539 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:17:00.214027 6539 webserver.cc:460] Webserver started at http://127.18.184.130:38657/ using document root <none> and password file <none>
I20260808 02:17:00.214570 6539 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:00.214630 6539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:00.214810 6539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:00.216925 6539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "47c16aa2374c4d17b036c3dd16d31382"
format_stamp: "Formatted at 2026-08-08 02:17:00 on dist-test-slave-9gcw"
server_key: "dc7d49c4ba939b546f5b5129b678fa4f"
server_key_iv: "2eb848288dd65ac6f38ff25e1b3fb94c"
server_key_version: "encryptionkey@0"
I20260808 02:17:00.217548 6539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "47c16aa2374c4d17b036c3dd16d31382"
format_stamp: "Formatted at 2026-08-08 02:17:00 on dist-test-slave-9gcw"
server_key: "dc7d49c4ba939b546f5b5129b678fa4f"
server_key_iv: "2eb848288dd65ac6f38ff25e1b3fb94c"
server_key_version: "encryptionkey@0"
I20260808 02:17:00.221596 6539 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:17:00.224342 6554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:00.225766 6539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:00.225906 6539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "47c16aa2374c4d17b036c3dd16d31382"
format_stamp: "Formatted at 2026-08-08 02:17:00 on dist-test-slave-9gcw"
server_key: "dc7d49c4ba939b546f5b5129b678fa4f"
server_key_iv: "2eb848288dd65ac6f38ff25e1b3fb94c"
server_key_version: "encryptionkey@0"
I20260808 02:17:00.226025 6539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:00.250653 6539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:00.254575 6539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:00.254804 6539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:00.255499 6539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:00.256430 6539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:00.256479 6539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:00.256520 6539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:00.256534 6539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:00.267387 6539 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:42901
I20260808 02:17:00.267496 6682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:42901 every 8 connection(s)
I20260808 02:17:00.268419 6539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:17:00.273025 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 6539
I20260808 02:17:00.273247 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:17:00.273563 19170 external_mini_cluster.cc:1428] Setting key f65763ee90b9b17e45717b039c52d065
Aug 08 02:17:00 dist-test-slave-9gcw krb5kdc[6269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155420, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:17:00.286513 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.270535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:58163 (local address 127.18.184.190:39301)
0808 02:17:00.270707 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.270711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.272061 (+ 1350us) server_negotiation.cc:407] Connection header received
0808 02:17:00.272982 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.272986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.273052 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.273210 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:00.274869 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.275597 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.277595 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.278013 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.278173 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.281530 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:00.281537 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:00.281541 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:00.281578 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:00.283640 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:00.284319 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:00.284326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:00.284330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:00.284457 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:00.284958 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:00.284964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:00.284968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:00.285286 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:00.285398 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.286117 (+ 719us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.286279 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:17:00.287388 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.270930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39301 (local address 127.18.184.130:58163)
0808 02:17:00.271881 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:00.271929 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:00.272725 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:00.273421 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:00.273429 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:00.273877 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:00.274594 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.274616 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.275935 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.275938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.277414 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.277427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.278248 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.278253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.278508 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.279324 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:00.279353 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:00.281284 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:00.283844 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:00.283851 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:00.283866 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:00.284178 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:00.284598 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:00.284601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:00.284603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:00.284837 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:00.285412 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:00.285418 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:00.285798 (+ 380us) client_negotiation.cc:769] Sending connection context
0808 02:17:00.286018 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 02:17:00.286335 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":554,"threads_started":1}
I20260808 02:17:00.288762 6683 heartbeater.cc:347] Connected to a master server at 127.18.184.190:39301
I20260808 02:17:00.289068 6683 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:00.289714 6683 heartbeater.cc:511] Master 127.18.184.190:39301 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
I20260808 02:17:00.291052 6312 ts_manager.cc:194] Registered new tserver with Master: 47c16aa2374c4d17b036c3dd16d31382 (127.18.184.130:42901)
I20260808 02:17:00.291930 6312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:58163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:17:00.308771 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:39301
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:00.428515 6689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:00.428830 6689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:00.428875 6689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:00.433324 6689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:00.433415 6689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:00.433478 6689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:17:00.439545 6689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:39301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:00.440737 6689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:00.441833 6689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:00.452930 6694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:00.452930 6697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:00.453008 6695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:00.453536 6689 server_base.cc:1034] running on GCE node
I20260808 02:17:00.454262 6689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:00.454989 6689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:00.456168 6689 hybrid_clock.cc:648] HybridClock initialized: now 1786155420456141 us; error 53 us; skew 500 ppm
Aug 08 02:17:00 dist-test-slave-9gcw krb5kdc[6269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155420, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:00.459389 6689 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:17:00.460404 6689 webserver.cc:460] Webserver started at http://127.18.184.131:40441/ using document root <none> and password file <none>
I20260808 02:17:00.461050 6689 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:00.461153 6689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:00.461444 6689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:00.463804 6689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "ef8690787c93434aa4cadee16980807a"
format_stamp: "Formatted at 2026-08-08 02:17:00 on dist-test-slave-9gcw"
server_key: "19e7dacb486c3fdda314e9ca9e8d5fde"
server_key_iv: "95d3121d4bd51a1572ea76de7ecb62e7"
server_key_version: "encryptionkey@0"
I20260808 02:17:00.464457 6689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "ef8690787c93434aa4cadee16980807a"
format_stamp: "Formatted at 2026-08-08 02:17:00 on dist-test-slave-9gcw"
server_key: "19e7dacb486c3fdda314e9ca9e8d5fde"
server_key_iv: "95d3121d4bd51a1572ea76de7ecb62e7"
server_key_version: "encryptionkey@0"
I20260808 02:17:00.468669 6689 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 02:17:00.471614 6704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:00.473098 6689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:00.473296 6689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "ef8690787c93434aa4cadee16980807a"
format_stamp: "Formatted at 2026-08-08 02:17:00 on dist-test-slave-9gcw"
server_key: "19e7dacb486c3fdda314e9ca9e8d5fde"
server_key_iv: "95d3121d4bd51a1572ea76de7ecb62e7"
server_key_version: "encryptionkey@0"
I20260808 02:17:00.473435 6689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:00.491629 6689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:00.496315 6689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:00.496584 6689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:00.497478 6689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:00.498693 6689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:00.498803 6689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:00.498895 6689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:00.498950 6689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:00.511157 6689 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:35827
I20260808 02:17:00.511229 6836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:35827 every 8 connection(s)
I20260808 02:17:00.512372 6689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:17:00.517853 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 6689
I20260808 02:17:00.518024 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:17:00.518394 19170 external_mini_cluster.cc:1428] Setting key 33cdf0e1624615f7893ec3e0b4a775f4
Aug 08 02:17:00 dist-test-slave-9gcw krb5kdc[6269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155420, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:17:00.529142 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.514875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:48705 (local address 127.18.184.190:39301)
0808 02:17:00.515059 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.515063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.516084 (+ 1021us) server_negotiation.cc:407] Connection header received
0808 02:17:00.517686 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.517690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.517743 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.517812 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:00.519507 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.520115 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.521617 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.521906 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.522023 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.524991 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:00.524995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:00.524997 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:00.525029 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:00.526574 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:00.527255 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:00.527259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:00.527267 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:00.527362 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:00.527778 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:00.527782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:00.527784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:00.528009 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:00.528138 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.528764 (+ 626us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.528925 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 02:17:00.530241 6839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.515399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:39301 (local address 127.18.184.131:48705)
0808 02:17:00.515902 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:00.515954 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:00.517390 (+ 1436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:00.518060 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:00.518072 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:00.518609 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:00.519325 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.519348 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.520274 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.520277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.521440 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.521451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.522084 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.522087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.522243 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.522934 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:00.522952 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:00.524761 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:00.526744 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:00.526752 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:00.526774 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:00.527116 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:00.527472 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:00.527477 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:00.527479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:00.527653 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:00.528138 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:00.528147 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:00.528478 (+ 331us) client_negotiation.cc:769] Sending connection context
0808 02:17:00.528705 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 02:17:00.529184 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":123,"threads_started":1}
I20260808 02:17:00.531831 6837 heartbeater.cc:347] Connected to a master server at 127.18.184.190:39301
I20260808 02:17:00.532310 6837 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:00.533025 6837 heartbeater.cc:511] Master 127.18.184.190:39301 requested a full tablet report, sending...
I20260808 02:17:00.534266 6312 ts_manager.cc:194] Registered new tserver with Master: ef8690787c93434aa4cadee16980807a (127.18.184.131:35827)
I20260808 02:17:00.534948 6312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:48705
I20260808 02:17:00.546008 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:17:00.557474 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.548208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54034 (local address 127.18.184.190:39301)
0808 02:17:00.548405 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.548410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.548491 (+ 81us) server_negotiation.cc:407] Connection header received
0808 02:17:00.548597 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.548599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.548649 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.548724 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:00.549537 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.550098 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.551063 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.551305 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.551393 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.553497 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:00.553500 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:00.553502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:00.553530 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:00.555277 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:00.555964 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:00.555967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:00.555969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:00.556058 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:00.556383 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:00.556385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:00.556386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:00.556614 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:00.556744 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.557036 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.557280 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:17:00.560945 6312 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:17:00.564311 6312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:00.581378 6850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.574312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:42901 (local address 127.0.0.1:59936)
0808 02:17:00.575666 (+ 1354us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:00.575680 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:00.575801 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:00.576523 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:00.576526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:00.576557 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:00.576810 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.576820 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.578453 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.578458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.579679 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.579689 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.580679 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.580683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.580809 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.580855 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 02:17:00.580984 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 02:17:00.581135 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":154,"spinlock_wait_cycles":34432,"thread_start_us":130,"threads_started":1}
I20260808 02:17:00.582024 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.575186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:32999 (local address 127.0.0.1:35038)
0808 02:17:00.575568 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:00.575584 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:00.575802 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:00.576466 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:00.576476 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:00.576507 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:00.576788 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.576799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.578475 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.578478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.580160 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.580173 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.581605 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.581611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.581729 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.581755 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:17:00.581831 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:17:00.581915 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":163,"spinlock_wait_cycles":4224,"thread_start_us":113,"threads_started":1}
I20260808 02:17:00.582257 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.574490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50546 (local address 127.18.184.131:35827)
0808 02:17:00.575028 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.575044 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.575623 (+ 579us) server_negotiation.cc:407] Connection header received
0808 02:17:00.575943 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.575949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.576170 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.576311 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:00.576895 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.578272 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.580333 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.581271 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.581403 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.581562 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.581953 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.582078 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":163,"threads_started":1}
I20260808 02:17:00.582526 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.575326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59936 (local address 127.18.184.130:42901)
0808 02:17:00.576085 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.576095 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.576115 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:17:00.576190 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.576194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.576342 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.576633 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:00.576950 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.578282 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.579820 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.580533 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.580669 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.580820 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.582106 (+ 1286us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.582370 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":124,"threads_started":1}
I20260808 02:17:00.582573 6854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.575641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.18.184.129:32999)
0808 02:17:00.576045 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.576052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.576073 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:17:00.576140 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.576144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.576299 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.576429 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:00.576952 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.578344 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.580319 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.581255 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.581394 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.581523 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.582246 (+ 723us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.582427 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":93,"threads_started":1}
I20260808 02:17:00.582726 6849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.574312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:35827 (local address 127.0.0.1:50546)
0808 02:17:00.575470 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:00.575529 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:00.575740 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:00.576512 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:00.576515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:00.576558 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:00.576777 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.576787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.578472 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.578476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.580161 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.580173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.581650 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.581654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.581755 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.581789 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:17:00.582471 (+ 682us) client_negotiation.cc:240] Negotiation successful
0808 02:17:00.582562 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"mutex_wait_us":255,"spinlock_wait_cycles":11136,"thread_start_us":99,"threads_started":1}
I20260808 02:17:00.585526 6459 tablet_service.cc:1467] Processing CreateTablet for tablet 837831866d8e4a349dd440e08fdc8e0b (DEFAULT_TABLE table=test-table [id=cb6bda0112d743c7a592963bffcd0fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:00.585616 6762 tablet_service.cc:1467] Processing CreateTablet for tablet 837831866d8e4a349dd440e08fdc8e0b (DEFAULT_TABLE table=test-table [id=cb6bda0112d743c7a592963bffcd0fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:00.585619 6610 tablet_service.cc:1467] Processing CreateTablet for tablet 837831866d8e4a349dd440e08fdc8e0b (DEFAULT_TABLE table=test-table [id=cb6bda0112d743c7a592963bffcd0fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:00.586678 6459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837831866d8e4a349dd440e08fdc8e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:00.586875 6610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837831866d8e4a349dd440e08fdc8e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:00.586913 6762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837831866d8e4a349dd440e08fdc8e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:00.593788 6855 tablet_bootstrap.cc:492] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce: Bootstrap starting.
I20260808 02:17:00.594982 6857 tablet_bootstrap.cc:492] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382: Bootstrap starting.
I20260808 02:17:00.595235 6858 tablet_bootstrap.cc:492] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a: Bootstrap starting.
I20260808 02:17:00.597409 6855 tablet_bootstrap.cc:654] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:00.597415 6858 tablet_bootstrap.cc:654] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:00.597415 6857 tablet_bootstrap.cc:654] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:00.598263 6855 log.cc:824] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:00.598826 6858 log.cc:824] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:00.598826 6857 log.cc:824] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:00.600612 6855 tablet_bootstrap.cc:492] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce: No bootstrap required, opened a new log
I20260808 02:17:00.600829 6855 ts_tablet_manager.cc:1397] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260808 02:17:00.603343 6857 tablet_bootstrap.cc:492] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382: No bootstrap required, opened a new log
I20260808 02:17:00.603646 6857 ts_tablet_manager.cc:1397] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260808 02:17:00.603756 6858 tablet_bootstrap.cc:492] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a: No bootstrap required, opened a new log
I20260808 02:17:00.603958 6858 ts_tablet_manager.cc:1397] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260808 02:17:00.605693 6855 raft_consensus.cc:348] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8690787c93434aa4cadee16980807a" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35827 } } peers { permanent_uuid: "47c16aa2374c4d17b036c3dd16d31382" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 42901 } } peers { permanent_uuid: "4b30008d5a8b4ed486c9d992a27813ce" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 32999 } }
I20260808 02:17:00.606021 6855 raft_consensus.cc:374] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:00.606096 6855 raft_consensus.cc:729] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b30008d5a8b4ed486c9d992a27813ce, State: Initialized, Role: FOLLOWER
I20260808 02:17:00.606599 6855 consensus_queue.cc:260] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8690787c93434aa4cadee16980807a" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35827 } } peers { permanent_uuid: "47c16aa2374c4d17b036c3dd16d31382" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 42901 } } peers { permanent_uuid: "4b30008d5a8b4ed486c9d992a27813ce" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 32999 } }
I20260808 02:17:00.607445 6534 heartbeater.cc:503] Master 127.18.184.190:39301 was elected leader, sending a full tablet report...
I20260808 02:17:00.607744 6855 ts_tablet_manager.cc:1428] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260808 02:17:00.607909 6858 raft_consensus.cc:348] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8690787c93434aa4cadee16980807a" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35827 } } peers { permanent_uuid: "47c16aa2374c4d17b036c3dd16d31382" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 42901 } } peers { permanent_uuid: "4b30008d5a8b4ed486c9d992a27813ce" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 32999 } }
I20260808 02:17:00.608186 6858 raft_consensus.cc:374] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:00.608242 6858 raft_consensus.cc:729] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8690787c93434aa4cadee16980807a, State: Initialized, Role: FOLLOWER
I20260808 02:17:00.608731 6858 consensus_queue.cc:260] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8690787c93434aa4cadee16980807a" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35827 } } peers { permanent_uuid: "47c16aa2374c4d17b036c3dd16d31382" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 42901 } } peers { permanent_uuid: "4b30008d5a8b4ed486c9d992a27813ce" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 32999 } }
I20260808 02:17:00.608239 6857 raft_consensus.cc:348] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8690787c93434aa4cadee16980807a" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35827 } } peers { permanent_uuid: "47c16aa2374c4d17b036c3dd16d31382" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 42901 } } peers { permanent_uuid: "4b30008d5a8b4ed486c9d992a27813ce" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 32999 } }
I20260808 02:17:00.609441 6857 raft_consensus.cc:374] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:00.609511 6857 raft_consensus.cc:729] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47c16aa2374c4d17b036c3dd16d31382, State: Initialized, Role: FOLLOWER
I20260808 02:17:00.609848 6837 heartbeater.cc:503] Master 127.18.184.190:39301 was elected leader, sending a full tablet report...
I20260808 02:17:00.609854 6858 ts_tablet_manager.cc:1428] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:17:00.609967 6857 consensus_queue.cc:260] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8690787c93434aa4cadee16980807a" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35827 } } peers { permanent_uuid: "47c16aa2374c4d17b036c3dd16d31382" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 42901 } } peers { permanent_uuid: "4b30008d5a8b4ed486c9d992a27813ce" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 32999 } }
I20260808 02:17:00.610778 6683 heartbeater.cc:503] Master 127.18.184.190:39301 was elected leader, sending a full tablet report...
I20260808 02:17:00.611343 6857 ts_tablet_manager.cc:1428] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260808 02:17:00.632844 6864 raft_consensus.cc:482] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:00.633224 6864 raft_consensus.cc:504] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8690787c93434aa4cadee16980807a" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35827 } } peers { permanent_uuid: "47c16aa2374c4d17b036c3dd16d31382" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 42901 } } peers { permanent_uuid: "4b30008d5a8b4ed486c9d992a27813ce" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 32999 } }
I20260808 02:17:00.634706 6864 leader_election.cc:288] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef8690787c93434aa4cadee16980807a (127.18.184.131:35827), 4b30008d5a8b4ed486c9d992a27813ce (127.18.184.129:32999)
I20260808 02:17:00.639215 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.634953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:35827 (local address 127.18.184.130:43069)
0808 02:17:00.635139 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:00.635161 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:00.635285 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:00.635652 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:00.635656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:00.635685 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:00.635863 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.635875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.636971 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.636974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.637962 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.637973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.638738 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.638741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.638864 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.638884 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:17:00.638977 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 02:17:00.639057 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 02:17:00.639205 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.635118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:43069 (local address 127.18.184.131:35827)
0808 02:17:00.635323 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.635329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.635348 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:17:00.635432 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.635438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.635518 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.635599 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:00.636028 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.636801 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.638105 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.638610 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.638751 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.638812 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.638998 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.639072 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260808 02:17:00.639812 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.635127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:32999 (local address 127.18.184.130:34781)
0808 02:17:00.635655 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:00.635672 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:00.635782 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:00.636115 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:00.636118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:00.636136 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:00.636252 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.636258 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.637345 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.637348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.638319 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:00.638336 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.639185 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.639190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:00.639326 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.639365 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 02:17:00.639529 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 02:17:00.639625 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":148,"threads_started":1}
I20260808 02:17:00.639837 6854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.635327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:34781 (local address 127.18.184.129:32999)
0808 02:17:00.635522 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.635527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.635795 (+ 268us) server_negotiation.cc:407] Connection header received
0808 02:17:00.635877 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.635879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.635921 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.636057 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:00.636375 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.637218 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.638488 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.639016 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.639196 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.639260 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.639582 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.639703 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 02:17:00.640095 6786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837831866d8e4a349dd440e08fdc8e0b" candidate_uuid: "47c16aa2374c4d17b036c3dd16d31382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8690787c93434aa4cadee16980807a" is_pre_election: true
I20260808 02:17:00.640492 6786 raft_consensus.cc:2454] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47c16aa2374c4d17b036c3dd16d31382 in term 0.
I20260808 02:17:00.640662 6483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837831866d8e4a349dd440e08fdc8e0b" candidate_uuid: "47c16aa2374c4d17b036c3dd16d31382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b30008d5a8b4ed486c9d992a27813ce" is_pre_election: true
I20260808 02:17:00.641058 6483 raft_consensus.cc:2454] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47c16aa2374c4d17b036c3dd16d31382 in term 0.
I20260808 02:17:00.641134 6556 leader_election.cc:302] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47c16aa2374c4d17b036c3dd16d31382, ef8690787c93434aa4cadee16980807a; no voters:
I20260808 02:17:00.641543 6864 raft_consensus.cc:2781] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:00.641642 6864 raft_consensus.cc:482] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:00.641697 6864 raft_consensus.cc:3037] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:00.643024 6864 raft_consensus.cc:504] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8690787c93434aa4cadee16980807a" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35827 } } peers { permanent_uuid: "47c16aa2374c4d17b036c3dd16d31382" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 42901 } } peers { permanent_uuid: "4b30008d5a8b4ed486c9d992a27813ce" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 32999 } }
I20260808 02:17:00.643440 6864 leader_election.cc:288] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [CANDIDATE]: Term 1 election: Requested vote from peers ef8690787c93434aa4cadee16980807a (127.18.184.131:35827), 4b30008d5a8b4ed486c9d992a27813ce (127.18.184.129:32999)
I20260808 02:17:00.643882 6483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837831866d8e4a349dd440e08fdc8e0b" candidate_uuid: "47c16aa2374c4d17b036c3dd16d31382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b30008d5a8b4ed486c9d992a27813ce"
I20260808 02:17:00.643929 6786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837831866d8e4a349dd440e08fdc8e0b" candidate_uuid: "47c16aa2374c4d17b036c3dd16d31382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8690787c93434aa4cadee16980807a"
I20260808 02:17:00.644039 6483 raft_consensus.cc:3037] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:00.644066 6786 raft_consensus.cc:3037] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:00.645375 6786 raft_consensus.cc:2454] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47c16aa2374c4d17b036c3dd16d31382 in term 1.
I20260808 02:17:00.645390 6483 raft_consensus.cc:2454] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47c16aa2374c4d17b036c3dd16d31382 in term 1.
I20260808 02:17:00.645768 6556 leader_election.cc:302] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47c16aa2374c4d17b036c3dd16d31382, ef8690787c93434aa4cadee16980807a; no voters:
I20260808 02:17:00.645989 6864 raft_consensus.cc:2781] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:00.646224 6864 raft_consensus.cc:686] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 1 LEADER]: Becoming Leader. State: Replica: 47c16aa2374c4d17b036c3dd16d31382, State: Running, Role: LEADER
I20260808 02:17:00.646678 6864 consensus_queue.cc:237] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8690787c93434aa4cadee16980807a" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35827 } } peers { permanent_uuid: "47c16aa2374c4d17b036c3dd16d31382" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 42901 } } peers { permanent_uuid: "4b30008d5a8b4ed486c9d992a27813ce" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 32999 } }
I20260808 02:17:00.650307 6317 catalog_manager.cc:5495] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47c16aa2374c4d17b036c3dd16d31382 (127.18.184.130). New cstate: current_term: 1 leader_uuid: "47c16aa2374c4d17b036c3dd16d31382" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8690787c93434aa4cadee16980807a" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47c16aa2374c4d17b036c3dd16d31382" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 42901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b30008d5a8b4ed486c9d992a27813ce" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 32999 } health_report { overall_health: UNKNOWN } } }
I20260808 02:17:00.667599 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.663675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59944 (local address 127.18.184.130:42901)
0808 02:17:00.663819 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.663824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.663844 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:17:00.663922 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.663924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.663983 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.664079 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:00.664381 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.664936 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.665930 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.666150 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.666258 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.666672 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:00.667152 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:00.667297 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.667410 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.667451 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:17:00.674463 6483 raft_consensus.cc:1261] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce [term 1 FOLLOWER]: Refusing update from remote peer 47c16aa2374c4d17b036c3dd16d31382: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:00.674461 6786 raft_consensus.cc:1261] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a [term 1 FOLLOWER]: Refusing update from remote peer 47c16aa2374c4d17b036c3dd16d31382: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:00.675228 6864 consensus_queue.cc:1035] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b30008d5a8b4ed486c9d992a27813ce" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 32999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:00.675474 6866 consensus_queue.cc:1035] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef8690787c93434aa4cadee16980807a" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:00.684572 6870 mvcc.cc:205] Tried to move back new op lower bound from 7316092603074007040 to 7316092602971856896. Current Snapshot: MvccSnapshot[applied={T|T < 7316092603074007040}]
I20260808 02:17:00.686306 6872 mvcc.cc:205] Tried to move back new op lower bound from 7316092603074007040 to 7316092602971856896. Current Snapshot: MvccSnapshot[applied={T|T < 7316092603074007040}]
I20260808 02:17:00.693480 6318 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:00.693718 6318 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:00.696342 6318 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6a6d2b285674e89837e5c282c4b0643: Sending DeleteTablet for 3 replicas of tablet 837831866d8e4a349dd440e08fdc8e0b
I20260808 02:17:00.697276 6459 tablet_service.cc:1514] Processing DeleteTablet for tablet 837831866d8e4a349dd440e08fdc8e0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:00 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:35038
I20260808 02:17:00.697271 6610 tablet_service.cc:1514] Processing DeleteTablet for tablet 837831866d8e4a349dd440e08fdc8e0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:00 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:59936
I20260808 02:17:00.697263 6762 tablet_service.cc:1514] Processing DeleteTablet for tablet 837831866d8e4a349dd440e08fdc8e0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:00 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:50546
I20260808 02:17:00.697917 6880 tablet_replica.cc:330] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce: stopping tablet replica
I20260808 02:17:00.698161 6881 tablet_replica.cc:330] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382: stopping tablet replica
I20260808 02:17:00.698336 6880 raft_consensus.cc:2229] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:00.698428 6881 raft_consensus.cc:2229] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:17:00.698671 6880 raft_consensus.cc:2258] T 837831866d8e4a349dd440e08fdc8e0b P 4b30008d5a8b4ed486c9d992a27813ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:00.698812 6881 raft_consensus.cc:2258] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:00.699056 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 6388
I20260808 02:17:00.699183 6882 tablet_replica.cc:330] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a: stopping tablet replica
I20260808 02:17:00.699494 6882 raft_consensus.cc:2229] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:00.699774 6882 raft_consensus.cc:2258] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:00.701007 6881 ts_tablet_manager.cc:1905] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:00.702589 6882 ts_tablet_manager.cc:1905] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:00.704499 6881 ts_tablet_manager.cc:1918] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:17:00.704622 6881 log.cc:1196] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/837831866d8e4a349dd440e08fdc8e0b
I20260808 02:17:00.705010 6881 ts_tablet_manager.cc:1939] T 837831866d8e4a349dd440e08fdc8e0b P 47c16aa2374c4d17b036c3dd16d31382: Deleting consensus metadata
I20260808 02:17:00.705314 6882 ts_tablet_manager.cc:1918] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:17:00.705418 6882 log.cc:1196] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/837831866d8e4a349dd440e08fdc8e0b
I20260808 02:17:00.705747 6882 ts_tablet_manager.cc:1939] T 837831866d8e4a349dd440e08fdc8e0b P ef8690787c93434aa4cadee16980807a: Deleting consensus metadata
I20260808 02:17:00.706502 6301 catalog_manager.cc:4841] TS 47c16aa2374c4d17b036c3dd16d31382 (127.18.184.130:42901): tablet 837831866d8e4a349dd440e08fdc8e0b (table test-table [id=cb6bda0112d743c7a592963bffcd0fcf]) successfully deleted
I20260808 02:17:00.706928 6302 catalog_manager.cc:4841] TS ef8690787c93434aa4cadee16980807a (127.18.184.131:35827): tablet 837831866d8e4a349dd440e08fdc8e0b (table test-table [id=cb6bda0112d743c7a592963bffcd0fcf]) successfully deleted
W20260808 02:17:00.712270 6301 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:32999 (error 108)
I20260808 02:17:00.712477 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 6539
I20260808 02:17:00.714690 6849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.714351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:32999 (local address 127.0.0.1:35052)
0808 02:17:00.714536 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:00.714601 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:32999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260808 02:17:00.714861 6301 catalog_manager.cc:4584] TS 4b30008d5a8b4ed486c9d992a27813ce (127.18.184.129:32999): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 837831866d8e4a349dd440e08fdc8e0b: Network error: Client connection negotiation failed: client connection to 127.18.184.129:32999: connect: Connection refused (error 111)
I20260808 02:17:00.722198 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 6689
I20260808 02:17:00.732554 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 6282
2026-08-08T02:17:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2522 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T02:17:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:00Z Disabled control of system clock
I20260808 02:17:00.769452 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:36307
--webserver_interface=127.18.184.190
--webserver_port=0
--builtin_ntp_servers=127.18.184.148:40229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:36307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:17:00.891664 6887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:00.891979 6887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:00.892019 6887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:00.896271 6887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:00.896343 6887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:00.896359 6887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:00.896377 6887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:00.896392 6887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:00.901484 6887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:40229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:36307
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:36307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:00.902642 6887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:00.904081 6887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:00.913425 6895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:00.913419 6893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:00.913437 6892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:00.914009 6887 server_base.cc:1034] running on GCE node
I20260808 02:17:00.914602 6887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:00.915840 6887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:00.917022 6887 hybrid_clock.cc:648] HybridClock initialized: now 1786155420917002 us; error 34 us; skew 500 ppm
I20260808 02:17:00.919492 6887 webserver.cc:460] Webserver started at http://127.18.184.190:39805/ using document root <none> and password file <none>
I20260808 02:17:00.920310 6887 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:00.920416 6887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:00.920676 6887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:00.923872 6887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "b05b4a455a234e61b289ce101572e883"
format_stamp: "Formatted at 2026-08-08 02:17:00 on dist-test-slave-9gcw"
server_key: "b0bc3f0564121eb4db8884dfd320d7d7"
server_key_iv: "cd65373eb747442225ea1311eb8bce64"
server_key_version: "encryptionkey@0"
I20260808 02:17:00.924649 6887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "b05b4a455a234e61b289ce101572e883"
format_stamp: "Formatted at 2026-08-08 02:17:00 on dist-test-slave-9gcw"
server_key: "b0bc3f0564121eb4db8884dfd320d7d7"
server_key_iv: "cd65373eb747442225ea1311eb8bce64"
server_key_version: "encryptionkey@0"
I20260808 02:17:00.929875 6887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 02:17:00.932797 6905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:00.934242 6887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:00.934412 6887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "b05b4a455a234e61b289ce101572e883"
format_stamp: "Formatted at 2026-08-08 02:17:00 on dist-test-slave-9gcw"
server_key: "b0bc3f0564121eb4db8884dfd320d7d7"
server_key_iv: "cd65373eb747442225ea1311eb8bce64"
server_key_version: "encryptionkey@0"
I20260808 02:17:00.934543 6887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:00.964165 6887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:00.965094 6887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:00.965349 6887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:00.973047 6887 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:36307
I20260808 02:17:00.973057 6963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:36307 every 8 connection(s)
I20260808 02:17:00.974141 6887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:00.977481 6964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:00.977614 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 6887
I20260808 02:17:00.977756 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:00.978056 19170 external_mini_cluster.cc:1428] Setting key 9a96152f4e38349ef1a2aef5f90afdfd
I20260808 02:17:00.984987 6964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883: Bootstrap starting.
I20260808 02:17:00.987439 6964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:00.987738 6967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.979453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50138 (local address 127.18.184.190:36307)
0808 02:17:00.980190 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.980213 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.980267 (+ 54us) server_negotiation.cc:407] Connection header received
0808 02:17:00.981126 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.981195 (+ 69us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.981645 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.982002 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:00.982441 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.983427 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.984321 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.984795 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.984980 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.985938 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:00.985943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:00.985956 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:00.986340 (+ 384us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:00.986396 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:00.986416 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:00.986527 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.986644 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.986952 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":128,"threads_started":1}
I20260808 02:17:00.988286 6964 log.cc:824] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:00.990675 6964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883: No bootstrap required, opened a new log
I20260808 02:17:00.994503 6964 raft_consensus.cc:348] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b05b4a455a234e61b289ce101572e883" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36307 } }
I20260808 02:17:00.994753 6964 raft_consensus.cc:374] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:00.994831 6964 raft_consensus.cc:729] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b05b4a455a234e61b289ce101572e883, State: Initialized, Role: FOLLOWER
I20260808 02:17:00.995251 6964 consensus_queue.cc:260] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b05b4a455a234e61b289ce101572e883" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36307 } }
I20260808 02:17:00.995438 6964 raft_consensus.cc:388] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:00.995512 6964 raft_consensus.cc:482] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:00.995612 6964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:00.996841 6964 raft_consensus.cc:504] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b05b4a455a234e61b289ce101572e883" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36307 } }
I20260808 02:17:00.997202 6964 leader_election.cc:302] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b05b4a455a234e61b289ce101572e883; no voters:
I20260808 02:17:00.997527 6964 leader_election.cc:288] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:00.997632 6969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:00.997952 6969 raft_consensus.cc:686] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [term 1 LEADER]: Becoming Leader. State: Replica: b05b4a455a234e61b289ce101572e883, State: Running, Role: LEADER
I20260808 02:17:00.998292 6969 consensus_queue.cc:237] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b05b4a455a234e61b289ce101572e883" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36307 } }
I20260808 02:17:00.999178 6964 sys_catalog.cc:564] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:00.999945 6971 sys_catalog.cc:455] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b05b4a455a234e61b289ce101572e883" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b05b4a455a234e61b289ce101572e883" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36307 } } }
I20260808 02:17:01.000088 6971 sys_catalog.cc:458] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:01.000653 6972 sys_catalog.cc:455] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b05b4a455a234e61b289ce101572e883. Latest consensus state: current_term: 1 leader_uuid: "b05b4a455a234e61b289ce101572e883" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b05b4a455a234e61b289ce101572e883" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36307 } } }
I20260808 02:17:01.000773 6972 sys_catalog.cc:458] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:01.001612 6977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:01.004657 6977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:01.010576 6977 catalog_manager.cc:1347] Generated new cluster ID: 75d9ec4019ec41ab810e137b5d61f0f7
I20260808 02:17:01.010661 6977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:01.029125 6977 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:01.030251 6977 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:01.036911 6977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883: Generated new TSK 0
I20260808 02:17:01.037793 6977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:01.052563 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36307
--builtin_ntp_servers=127.18.184.148:40229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:17:01.173219 6990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:01.173511 6990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:01.173550 6990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:01.177685 6990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:01.177749 6990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:01.177851 6990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:17:01.183235 6990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:40229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:01.184582 6990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:01.186139 6990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:01.196738 6995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:01.196774 7001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:01.196831 6998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:01.196959 6990 server_base.cc:1034] running on GCE node
I20260808 02:17:01.197600 6990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:01.198410 6990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:01.199589 6990 hybrid_clock.cc:648] HybridClock initialized: now 1786155421199565 us; error 36 us; skew 500 ppm
I20260808 02:17:01.202049 6990 webserver.cc:460] Webserver started at http://127.18.184.129:33073/ using document root <none> and password file <none>
I20260808 02:17:01.202768 6990 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:01.202877 6990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:01.203140 6990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:01.205602 6990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "b0076b74a4304258a32008d5499399c4"
format_stamp: "Formatted at 2026-08-08 02:17:01 on dist-test-slave-9gcw"
server_key: "c38f359ab3b2f53280a03a6d1a253180"
server_key_iv: "4cdbf30ce50cee1f6cf9b9dfe66c5ca1"
server_key_version: "encryptionkey@0"
I20260808 02:17:01.206265 6990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "b0076b74a4304258a32008d5499399c4"
format_stamp: "Formatted at 2026-08-08 02:17:01 on dist-test-slave-9gcw"
server_key: "c38f359ab3b2f53280a03a6d1a253180"
server_key_iv: "4cdbf30ce50cee1f6cf9b9dfe66c5ca1"
server_key_version: "encryptionkey@0"
I20260808 02:17:01.210556 6990 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 02:17:01.213622 7008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:01.215083 6990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:17:01.215263 6990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "b0076b74a4304258a32008d5499399c4"
format_stamp: "Formatted at 2026-08-08 02:17:01 on dist-test-slave-9gcw"
server_key: "c38f359ab3b2f53280a03a6d1a253180"
server_key_iv: "4cdbf30ce50cee1f6cf9b9dfe66c5ca1"
server_key_version: "encryptionkey@0"
I20260808 02:17:01.215437 6990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:01.228193 6990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:01.229004 6990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:01.229308 6990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:01.229974 6990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:01.231045 6990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:01.231122 6990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:01.231192 6990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:01.231243 6990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:01.241660 6990 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:40045
I20260808 02:17:01.241681 7135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:40045 every 8 connection(s)
I20260808 02:17:01.242686 6990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:17:01.250902 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 6990
I20260808 02:17:01.251075 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:17:01.251478 19170 external_mini_cluster.cc:1428] Setting key e9a51fb09998df18aa8a1047300f1baa
I20260808 02:17:01.253095 6967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.244778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:60491 (local address 127.18.184.190:36307)
0808 02:17:01.244955 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:01.244959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:01.246099 (+ 1140us) server_negotiation.cc:407] Connection header received
0808 02:17:01.247642 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:01.247645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:01.247710 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:01.247815 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:01.248761 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.249342 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.250523 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.250796 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.250903 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.252581 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:01.252583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:01.252585 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:01.252596 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:01.252615 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:01.252623 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:01.252682 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:01.252877 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 02:17:01.252947 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 02:17:01.253970 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36307
--builtin_ntp_servers=127.18.184.148:40229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:17:01.254175 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.245208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:36307 (local address 127.18.184.129:60491)
0808 02:17:01.245935 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:01.245979 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:01.247399 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:01.248018 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:01.248025 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:01.248113 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:01.248613 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:01.248625 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.249478 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.249481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:01.250388 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:01.250398 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.251119 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.251128 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:01.251380 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.252263 (+ 883us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:17:01.252265 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:01.252332 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:17:01.252404 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:01.252712 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:01.252721 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:01.252728 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:17:01.252868 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 02:17:01.253244 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":136,"threads_started":1}
I20260808 02:17:01.255541 7136 heartbeater.cc:347] Connected to a master server at 127.18.184.190:36307
I20260808 02:17:01.255851 7136 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:01.256565 7136 heartbeater.cc:511] Master 127.18.184.190:36307 requested a full tablet report, sending...
I20260808 02:17:01.258229 6924 ts_manager.cc:194] Registered new tserver with Master: b0076b74a4304258a32008d5499399c4 (127.18.184.129:40045)
I20260808 02:17:01.259506 6924 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.184.129:60491
W20260808 02:17:01.374716 7140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:01.375026 7140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:01.375068 7140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:01.379402 7140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:01.379485 7140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:01.379545 7140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:17:01.384872 7140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:40229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:01.386148 7140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:01.387688 7140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:01.398504 7148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:01.398563 7146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:01.398556 7140 server_base.cc:1034] running on GCE node
W20260808 02:17:01.398504 7145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:01.399343 7140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:01.400218 7140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:01.401419 7140 hybrid_clock.cc:648] HybridClock initialized: now 1786155421401389 us; error 36 us; skew 500 ppm
I20260808 02:17:01.403607 7140 webserver.cc:460] Webserver started at http://127.18.184.130:44171/ using document root <none> and password file <none>
I20260808 02:17:01.404285 7140 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:01.404390 7140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:01.404627 7140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:01.406956 7140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "0b3f600faa5d4612b64f2a9084de80aa"
format_stamp: "Formatted at 2026-08-08 02:17:01 on dist-test-slave-9gcw"
server_key: "fe3fc2c04c31d73bdd19b313415ca5a5"
server_key_iv: "7100581291707c7aabb61cd7a3ff7df0"
server_key_version: "encryptionkey@0"
I20260808 02:17:01.407653 7140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "0b3f600faa5d4612b64f2a9084de80aa"
format_stamp: "Formatted at 2026-08-08 02:17:01 on dist-test-slave-9gcw"
server_key: "fe3fc2c04c31d73bdd19b313415ca5a5"
server_key_iv: "7100581291707c7aabb61cd7a3ff7df0"
server_key_version: "encryptionkey@0"
I20260808 02:17:01.411979 7140 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 02:17:01.414867 7155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:01.416404 7140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:17:01.416558 7140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "0b3f600faa5d4612b64f2a9084de80aa"
format_stamp: "Formatted at 2026-08-08 02:17:01 on dist-test-slave-9gcw"
server_key: "fe3fc2c04c31d73bdd19b313415ca5a5"
server_key_iv: "7100581291707c7aabb61cd7a3ff7df0"
server_key_version: "encryptionkey@0"
I20260808 02:17:01.416741 7140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:01.440574 7140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:01.441413 7140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:01.441634 7140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:01.442317 7140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:01.443401 7140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:01.443450 7140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:01.443523 7140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:01.443568 7140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:01.452908 7140 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:38655
I20260808 02:17:01.452988 7285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:38655 every 8 connection(s)
I20260808 02:17:01.453982 7140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:17:01.463248 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 7140
I20260808 02:17:01.463420 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:17:01.463759 19170 external_mini_cluster.cc:1428] Setting key d415e8ea661bfd11f73399396b768f8f
I20260808 02:17:01.464654 6967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.456110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:53351 (local address 127.18.184.190:36307)
0808 02:17:01.456260 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:01.456264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:01.457389 (+ 1125us) server_negotiation.cc:407] Connection header received
0808 02:17:01.458939 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:01.458943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:01.459009 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:01.459123 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:01.460066 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.460896 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.462237 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.462521 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.462650 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.464092 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:01.464096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:01.464099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:01.464116 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:01.464143 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:01.464152 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:01.464234 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:01.464417 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 02:17:01.464489 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:17:01.465768 7288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.456457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:36307 (local address 127.18.184.130:53351)
0808 02:17:01.457216 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:01.457280 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:01.458713 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:01.459316 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:01.459324 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:01.459409 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:01.459913 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:01.459925 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.461062 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.461066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:01.462067 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:01.462078 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.462693 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.462697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:01.462880 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.463752 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:17:01.463753 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:01.463822 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:17:01.463918 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:01.464244 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:01.464261 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:01.464267 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:17:01.464417 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 02:17:01.464823 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":131,"threads_started":1}
I20260808 02:17:01.466061 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36307
--builtin_ntp_servers=127.18.184.148:40229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:17:01.467234 7286 heartbeater.cc:347] Connected to a master server at 127.18.184.190:36307
I20260808 02:17:01.467519 7286 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:01.468209 7286 heartbeater.cc:511] Master 127.18.184.190:36307 requested a full tablet report, sending...
I20260808 02:17:01.469661 6924 ts_manager.cc:194] Registered new tserver with Master: 0b3f600faa5d4612b64f2a9084de80aa (127.18.184.130:38655)
I20260808 02:17:01.470547 6924 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.184.130:53351
W20260808 02:17:01.589020 7289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:01.589411 7289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:01.589453 7289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:01.593683 7289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:01.593755 7289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:01.593815 7289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:17:01.598948 7289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:40229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:36307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:01.600188 7289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:01.601759 7289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:01.611856 7294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:01.611846 7295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:01.612175 7289 server_base.cc:1034] running on GCE node
W20260808 02:17:01.612391 7297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:01.612854 7289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:01.613608 7289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:01.614806 7289 hybrid_clock.cc:648] HybridClock initialized: now 1786155421614787 us; error 31 us; skew 500 ppm
I20260808 02:17:01.616860 7289 webserver.cc:460] Webserver started at http://127.18.184.131:39257/ using document root <none> and password file <none>
I20260808 02:17:01.617602 7289 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:01.617676 7289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:01.617947 7289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:01.620468 7289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "2f4b1f6065b94876b9186d01db246064"
format_stamp: "Formatted at 2026-08-08 02:17:01 on dist-test-slave-9gcw"
server_key: "8ee63aa6a39cf4422dea5c247b007c94"
server_key_iv: "5ec0285796d9bf60c78079c82ed739c9"
server_key_version: "encryptionkey@0"
I20260808 02:17:01.621325 7289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "2f4b1f6065b94876b9186d01db246064"
format_stamp: "Formatted at 2026-08-08 02:17:01 on dist-test-slave-9gcw"
server_key: "8ee63aa6a39cf4422dea5c247b007c94"
server_key_iv: "5ec0285796d9bf60c78079c82ed739c9"
server_key_version: "encryptionkey@0"
I20260808 02:17:01.625838 7289 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:17:01.628638 7303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:01.630045 7289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:01.630219 7289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "2f4b1f6065b94876b9186d01db246064"
format_stamp: "Formatted at 2026-08-08 02:17:01 on dist-test-slave-9gcw"
server_key: "8ee63aa6a39cf4422dea5c247b007c94"
server_key_iv: "5ec0285796d9bf60c78079c82ed739c9"
server_key_version: "encryptionkey@0"
I20260808 02:17:01.630357 7289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:01.652716 7289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:01.653822 7289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:01.654079 7289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:01.655032 7289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:01.656208 7289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:01.656258 7289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:01.656304 7289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:01.656380 7289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:01.667039 7289 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:35261
I20260808 02:17:01.667094 7436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:35261 every 8 connection(s)
I20260808 02:17:01.668077 7289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:17:01.676097 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 7289
I20260808 02:17:01.676252 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:17:01.676594 19170 external_mini_cluster.cc:1428] Setting key a4cc108c89b6de6807c0760e512a56be
I20260808 02:17:01.680583 6967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.670094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:43811 (local address 127.18.184.190:36307)
0808 02:17:01.670268 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:01.670272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:01.671320 (+ 1048us) server_negotiation.cc:407] Connection header received
0808 02:17:01.672870 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:01.672873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:01.672924 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:01.673006 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:01.674114 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.675019 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.678262 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.678582 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.678671 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.680069 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:01.680072 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:01.680074 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:01.680087 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:01.680106 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:01.680114 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:01.680181 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:01.680405 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 02:17:01.680462 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:17:01.681849 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.670431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:36307 (local address 127.18.184.131:43811)
0808 02:17:01.671134 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:01.671202 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:01.672641 (+ 1439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:01.673296 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:01.673304 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:01.673394 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:01.673908 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:01.673922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.675198 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.675202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:01.678119 (+ 2917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:01.678132 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.678771 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.678775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:01.679002 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.679774 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:17:01.679776 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:01.679829 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:17:01.679904 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:01.680247 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:01.680257 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:01.680263 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:17:01.680391 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 02:17:01.680809 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":119,"threads_started":1}
I20260808 02:17:01.683218 7437 heartbeater.cc:347] Connected to a master server at 127.18.184.190:36307
I20260808 02:17:01.683481 7437 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:01.684065 7437 heartbeater.cc:511] Master 127.18.184.190:36307 requested a full tablet report, sending...
I20260808 02:17:01.685458 6924 ts_manager.cc:194] Registered new tserver with Master: 2f4b1f6065b94876b9186d01db246064 (127.18.184.131:35261)
I20260808 02:17:01.686051 6924 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.184.131:43811
I20260808 02:17:01.691550 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:17:01.698468 6967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.694059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50148 (local address 127.18.184.190:36307)
0808 02:17:01.694212 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:01.694217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:01.694470 (+ 253us) server_negotiation.cc:407] Connection header received
0808 02:17:01.694581 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:01.694584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:01.694641 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:01.694746 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:01.695084 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.695681 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.696533 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.696756 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.696859 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.698009 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:01.698013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:01.698015 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:01.698028 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:01.698049 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:01.698059 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:01.698187 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:01.698307 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 02:17:01.698344 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:17:01.702284 6924 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:17:01.705482 6924 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:01.721570 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.714550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:38655 (local address 127.0.0.1:58702)
0808 02:17:01.715279 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:01.715333 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:01.715501 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:01.716578 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:01.716580 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:01.716608 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:01.716892 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:01.716899 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.718701 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.718704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:01.719839 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:01.719849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.720942 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.720946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:01.721019 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.721096 (+ 77us) client_negotiation.cc:769] Sending connection context
0808 02:17:01.721219 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 02:17:01.721370 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":77,"spinlock_wait_cycles":20864,"thread_start_us":122,"threads_started":1}
I20260808 02:17:01.721570 7451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.714844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:35261 (local address 127.0.0.1:42782)
0808 02:17:01.715295 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:01.715333 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:01.715499 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:01.716670 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:01.716673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:01.716692 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:01.716861 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:01.716868 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.717970 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.717977 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:01.719664 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:01.719674 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.720878 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.720883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:01.721016 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.721096 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 02:17:01.721339 (+ 243us) client_negotiation.cc:240] Negotiation successful
0808 02:17:01.721398 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":109,"spinlock_wait_cycles":29184,"thread_start_us":100,"threads_started":1}
I20260808 02:17:01.721675 7452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.715321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42782 (local address 127.18.184.131:35261)
0808 02:17:01.716027 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:01.716040 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:01.716067 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:17:01.716177 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:01.716183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:01.716361 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:01.716512 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:01.716998 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.717849 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.719888 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.720689 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.720892 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.721098 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:01.721403 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 02:17:01.721514 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":179,"threads_started":1}
I20260808 02:17:01.722168 7448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.714468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:40045 (local address 127.0.0.1:47804)
0808 02:17:01.715288 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:01.715334 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:01.715499 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:01.716567 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:01.716573 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:01.716609 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:01.716893 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:01.716899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.718335 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.718339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:01.719642 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:01.719653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.721190 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.721195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:01.721298 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.721318 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:17:01.721414 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 02:17:01.721489 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":216,"spinlock_wait_cycles":27008,"thread_start_us":96,"threads_started":1}
I20260808 02:17:01.722754 7450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.714588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.18.184.129:40045)
0808 02:17:01.715847 (+ 1259us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:01.715856 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:01.715885 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:17:01.715966 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:01.715971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:01.716184 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:01.716345 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:01.717038 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.718205 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.719776 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.720694 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.720892 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.721098 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:01.722484 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0808 02:17:01.722576 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1093,"thread_start_us":108,"threads_started":1}
I20260808 02:17:01.723449 7453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:01.715757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58702 (local address 127.18.184.130:38655)
0808 02:17:01.716197 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:01.716208 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:01.716227 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:17:01.716294 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:01.716298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:01.716449 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:01.716586 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:01.717038 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.718562 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.719941 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:01.720814 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:01.722948 (+ 2134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:01.723142 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:01.723189 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 02:17:01.723282 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":149,"threads_started":1}
I20260808 02:17:01.725108 7061 tablet_service.cc:1467] Processing CreateTablet for tablet 967e698772db41e19406e1889afc9c2e (DEFAULT_TABLE table=test-table [id=92632d28e0ea40dbb153cd975f203645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:01.725029 7358 tablet_service.cc:1467] Processing CreateTablet for tablet 967e698772db41e19406e1889afc9c2e (DEFAULT_TABLE table=test-table [id=92632d28e0ea40dbb153cd975f203645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:01.726294 7358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 967e698772db41e19406e1889afc9c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:01.726310 7061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 967e698772db41e19406e1889afc9c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:01.728560 7211 tablet_service.cc:1467] Processing CreateTablet for tablet 967e698772db41e19406e1889afc9c2e (DEFAULT_TABLE table=test-table [id=92632d28e0ea40dbb153cd975f203645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:01.729755 7211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 967e698772db41e19406e1889afc9c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:01.732975 7454 tablet_bootstrap.cc:492] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4: Bootstrap starting.
I20260808 02:17:01.735517 7454 tablet_bootstrap.cc:654] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:01.736286 7454 log.cc:824] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:01.737032 7455 tablet_bootstrap.cc:492] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064: Bootstrap starting.
I20260808 02:17:01.738956 7454 tablet_bootstrap.cc:492] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4: No bootstrap required, opened a new log
I20260808 02:17:01.739228 7454 ts_tablet_manager.cc:1397] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:17:01.739241 7457 tablet_bootstrap.cc:492] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa: Bootstrap starting.
I20260808 02:17:01.739491 7455 tablet_bootstrap.cc:654] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:01.740252 7455 log.cc:824] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:01.741631 7457 tablet_bootstrap.cc:654] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:01.742233 7455 tablet_bootstrap.cc:492] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064: No bootstrap required, opened a new log
I20260808 02:17:01.742445 7455 ts_tablet_manager.cc:1397] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:17:01.742498 7457 log.cc:824] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:01.743790 7454 raft_consensus.cc:348] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0076b74a4304258a32008d5499399c4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40045 } } peers { permanent_uuid: "0b3f600faa5d4612b64f2a9084de80aa" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38655 } } peers { permanent_uuid: "2f4b1f6065b94876b9186d01db246064" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35261 } }
I20260808 02:17:01.744107 7454 raft_consensus.cc:374] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:01.744181 7454 raft_consensus.cc:729] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0076b74a4304258a32008d5499399c4, State: Initialized, Role: FOLLOWER
I20260808 02:17:01.744727 7454 consensus_queue.cc:260] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0076b74a4304258a32008d5499399c4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40045 } } peers { permanent_uuid: "0b3f600faa5d4612b64f2a9084de80aa" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38655 } } peers { permanent_uuid: "2f4b1f6065b94876b9186d01db246064" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35261 } }
I20260808 02:17:01.744848 7457 tablet_bootstrap.cc:492] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa: No bootstrap required, opened a new log
I20260808 02:17:01.745038 7457 ts_tablet_manager.cc:1397] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:17:01.745868 7454 ts_tablet_manager.cc:1428] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 02:17:01.746285 7136 heartbeater.cc:503] Master 127.18.184.190:36307 was elected leader, sending a full tablet report...
I20260808 02:17:01.746799 7455 raft_consensus.cc:348] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0076b74a4304258a32008d5499399c4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40045 } } peers { permanent_uuid: "0b3f600faa5d4612b64f2a9084de80aa" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38655 } } peers { permanent_uuid: "2f4b1f6065b94876b9186d01db246064" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35261 } }
I20260808 02:17:01.747057 7455 raft_consensus.cc:374] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:01.747112 7455 raft_consensus.cc:729] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f4b1f6065b94876b9186d01db246064, State: Initialized, Role: FOLLOWER
I20260808 02:17:01.747603 7455 consensus_queue.cc:260] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0076b74a4304258a32008d5499399c4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40045 } } peers { permanent_uuid: "0b3f600faa5d4612b64f2a9084de80aa" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38655 } } peers { permanent_uuid: "2f4b1f6065b94876b9186d01db246064" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35261 } }
I20260808 02:17:01.748288 7437 heartbeater.cc:503] Master 127.18.184.190:36307 was elected leader, sending a full tablet report...
I20260808 02:17:01.748654 7455 ts_tablet_manager.cc:1428] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260808 02:17:01.749011 7457 raft_consensus.cc:348] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0076b74a4304258a32008d5499399c4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40045 } } peers { permanent_uuid: "0b3f600faa5d4612b64f2a9084de80aa" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38655 } } peers { permanent_uuid: "2f4b1f6065b94876b9186d01db246064" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35261 } }
I20260808 02:17:01.749336 7457 raft_consensus.cc:374] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:01.749398 7457 raft_consensus.cc:729] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b3f600faa5d4612b64f2a9084de80aa, State: Initialized, Role: FOLLOWER
I20260808 02:17:01.749910 7457 consensus_queue.cc:260] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0076b74a4304258a32008d5499399c4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40045 } } peers { permanent_uuid: "0b3f600faa5d4612b64f2a9084de80aa" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38655 } } peers { permanent_uuid: "2f4b1f6065b94876b9186d01db246064" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35261 } }
I20260808 02:17:01.750710 7286 heartbeater.cc:503] Master 127.18.184.190:36307 was elected leader, sending a full tablet report...
I20260808 02:17:01.751014 7457 ts_tablet_manager.cc:1428] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260808 02:17:01.919114 7438 tablet.cc:2369] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:17:01.955284 7287 tablet.cc:2369] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:17:01.994192 7138 tablet.cc:2369] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:17:02.042689 7460 raft_consensus.cc:482] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:02.042965 7460 raft_consensus.cc:504] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0076b74a4304258a32008d5499399c4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40045 } } peers { permanent_uuid: "0b3f600faa5d4612b64f2a9084de80aa" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38655 } } peers { permanent_uuid: "2f4b1f6065b94876b9186d01db246064" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35261 } }
I20260808 02:17:02.044279 7460 leader_election.cc:288] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b3f600faa5d4612b64f2a9084de80aa (127.18.184.130:38655), 2f4b1f6065b94876b9186d01db246064 (127.18.184.131:35261)
I20260808 02:17:02.048492 7453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:02.044648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:34525 (local address 127.18.184.130:38655)
0808 02:17:02.044857 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:02.044862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:02.044942 (+ 80us) server_negotiation.cc:407] Connection header received
0808 02:17:02.045222 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:02.045226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:02.045298 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:02.045373 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:02.045753 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:02.046302 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:02.047474 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:02.047866 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:02.048000 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:02.048055 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:02.048242 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 02:17:02.048326 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:17:02.048503 7452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:02.044893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:54413 (local address 127.18.184.131:35261)
0808 02:17:02.045025 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:02.045029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:02.045172 (+ 143us) server_negotiation.cc:407] Connection header received
0808 02:17:02.045344 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:02.045347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:02.045426 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:02.045508 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:02.045847 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:02.046376 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:02.047475 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:02.047879 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:02.048013 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:02.048062 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:02.048281 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 02:17:02.048350 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:17:02.048525 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:02.044513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:38655 (local address 127.18.184.129:34525)
0808 02:17:02.044842 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:02.044860 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:02.045033 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:02.045411 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:02.045415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:02.045433 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:02.045622 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:02.045635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:02.046425 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:02.046429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:02.047345 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:02.047354 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:02.048000 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:02.048004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:02.048104 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:02.048133 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:17:02.048196 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 02:17:02.048292 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":48,"thread_start_us":120,"threads_started":1}
I20260808 02:17:02.048769 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:02.044646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:35261 (local address 127.18.184.129:54413)
0808 02:17:02.045041 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:02.045053 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:02.045218 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:02.045539 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:02.045542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:02.045558 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:02.045714 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:02.045720 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:02.046491 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:02.046493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:02.047347 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:02.047356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:02.048022 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:02.048026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:02.048127 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:02.048145 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 02:17:02.048239 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 02:17:02.048311 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":31,"thread_start_us":129,"threads_started":1}
I20260808 02:17:02.049302 7232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "967e698772db41e19406e1889afc9c2e" candidate_uuid: "b0076b74a4304258a32008d5499399c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b3f600faa5d4612b64f2a9084de80aa" is_pre_election: true
I20260808 02:17:02.049592 7377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "967e698772db41e19406e1889afc9c2e" candidate_uuid: "b0076b74a4304258a32008d5499399c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f4b1f6065b94876b9186d01db246064" is_pre_election: true
I20260808 02:17:02.049643 7232 raft_consensus.cc:2454] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0076b74a4304258a32008d5499399c4 in term 0.
I20260808 02:17:02.049878 7377 raft_consensus.cc:2454] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0076b74a4304258a32008d5499399c4 in term 0.
I20260808 02:17:02.050240 7011 leader_election.cc:302] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b3f600faa5d4612b64f2a9084de80aa, b0076b74a4304258a32008d5499399c4; no voters:
I20260808 02:17:02.050549 7460 raft_consensus.cc:2781] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:02.050628 7460 raft_consensus.cc:482] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:02.050668 7460 raft_consensus.cc:3037] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:02.051828 7460 raft_consensus.cc:504] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0076b74a4304258a32008d5499399c4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40045 } } peers { permanent_uuid: "0b3f600faa5d4612b64f2a9084de80aa" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38655 } } peers { permanent_uuid: "2f4b1f6065b94876b9186d01db246064" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35261 } }
I20260808 02:17:02.052214 7460 leader_election.cc:288] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [CANDIDATE]: Term 1 election: Requested vote from peers 0b3f600faa5d4612b64f2a9084de80aa (127.18.184.130:38655), 2f4b1f6065b94876b9186d01db246064 (127.18.184.131:35261)
I20260808 02:17:02.052614 7377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "967e698772db41e19406e1889afc9c2e" candidate_uuid: "b0076b74a4304258a32008d5499399c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f4b1f6065b94876b9186d01db246064"
I20260808 02:17:02.052621 7232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "967e698772db41e19406e1889afc9c2e" candidate_uuid: "b0076b74a4304258a32008d5499399c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b3f600faa5d4612b64f2a9084de80aa"
I20260808 02:17:02.052770 7232 raft_consensus.cc:3037] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:02.052731 7377 raft_consensus.cc:3037] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:02.053944 7232 raft_consensus.cc:2454] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0076b74a4304258a32008d5499399c4 in term 1.
I20260808 02:17:02.053949 7377 raft_consensus.cc:2454] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0076b74a4304258a32008d5499399c4 in term 1.
I20260808 02:17:02.054303 7011 leader_election.cc:302] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b3f600faa5d4612b64f2a9084de80aa, b0076b74a4304258a32008d5499399c4; no voters:
I20260808 02:17:02.054560 7460 raft_consensus.cc:2781] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:02.054764 7460 raft_consensus.cc:686] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 1 LEADER]: Becoming Leader. State: Replica: b0076b74a4304258a32008d5499399c4, State: Running, Role: LEADER
I20260808 02:17:02.055166 7460 consensus_queue.cc:237] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0076b74a4304258a32008d5499399c4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40045 } } peers { permanent_uuid: "0b3f600faa5d4612b64f2a9084de80aa" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38655 } } peers { permanent_uuid: "2f4b1f6065b94876b9186d01db246064" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35261 } }
I20260808 02:17:02.058643 6924 catalog_manager.cc:5495] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0076b74a4304258a32008d5499399c4 (127.18.184.129). New cstate: current_term: 1 leader_uuid: "b0076b74a4304258a32008d5499399c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0076b74a4304258a32008d5499399c4" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 40045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b3f600faa5d4612b64f2a9084de80aa" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f4b1f6065b94876b9186d01db246064" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35261 } health_report { overall_health: UNKNOWN } } }
I20260808 02:17:02.089603 7450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:02.085840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47808 (local address 127.18.184.129:40045)
0808 02:17:02.085999 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:02.086006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:02.086025 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:17:02.086111 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:02.086114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:02.086171 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:02.086295 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:02.086557 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:02.087081 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:02.087981 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:02.088208 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:02.088331 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:02.088660 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:02.089081 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:02.089345 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:02.089407 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 02:17:02.089447 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:17:02.095907 7377 raft_consensus.cc:1261] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064 [term 1 FOLLOWER]: Refusing update from remote peer b0076b74a4304258a32008d5499399c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:02.095932 7232 raft_consensus.cc:1261] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa [term 1 FOLLOWER]: Refusing update from remote peer b0076b74a4304258a32008d5499399c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:02.096670 7465 consensus_queue.cc:1035] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f4b1f6065b94876b9186d01db246064" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 35261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:02.096877 7460 consensus_queue.cc:1035] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b3f600faa5d4612b64f2a9084de80aa" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:02.108283 7469 mvcc.cc:205] Tried to move back new op lower bound from 7316092608897253376 to 7316092608740900864. Current Snapshot: MvccSnapshot[applied={T|T < 7316092608897253376}]
I20260808 02:17:02.113268 6924 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:02.113461 6924 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:02.116076 6924 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b05b4a455a234e61b289ce101572e883: Sending DeleteTablet for 3 replicas of tablet 967e698772db41e19406e1889afc9c2e
I20260808 02:17:02.116925 7061 tablet_service.cc:1514] Processing DeleteTablet for tablet 967e698772db41e19406e1889afc9c2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:02 UTC) from {username='slave'} at 127.0.0.1:47804
I20260808 02:17:02.117100 7358 tablet_service.cc:1514] Processing DeleteTablet for tablet 967e698772db41e19406e1889afc9c2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:02 UTC) from {username='slave'} at 127.0.0.1:42782
I20260808 02:17:02.117074 7211 tablet_service.cc:1514] Processing DeleteTablet for tablet 967e698772db41e19406e1889afc9c2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:02 UTC) from {username='slave'} at 127.0.0.1:58702
I20260808 02:17:02.117571 7479 tablet_replica.cc:330] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4: stopping tablet replica
I20260808 02:17:02.118031 7481 tablet_replica.cc:330] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa: stopping tablet replica
I20260808 02:17:02.118089 7479 raft_consensus.cc:2229] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:17:02.118358 7481 raft_consensus.cc:2229] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:02.118606 7479 raft_consensus.cc:2258] T 967e698772db41e19406e1889afc9c2e P b0076b74a4304258a32008d5499399c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:02.118728 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 6990
I20260808 02:17:02.125229 7480 tablet_replica.cc:330] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064: stopping tablet replica
I20260808 02:17:02.125355 7481 raft_consensus.cc:2258] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:02.125691 7480 raft_consensus.cc:2229] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:02.126128 7480 raft_consensus.cc:2258] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:02.127345 7481 ts_tablet_manager.cc:1905] T 967e698772db41e19406e1889afc9c2e P 0b3f600faa5d4612b64f2a9084de80aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:02.127491 7480 ts_tablet_manager.cc:1905] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 02:17:02.128145 6910 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:40045 (error 108)
I20260808 02:17:02.129105 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 7140
I20260808 02:17:02.130716 7480 ts_tablet_manager.cc:1918] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:17:02.130827 7480 log.cc:1196] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/967e698772db41e19406e1889afc9c2e
I20260808 02:17:02.131227 7480 ts_tablet_manager.cc:1939] T 967e698772db41e19406e1889afc9c2e P 2f4b1f6065b94876b9186d01db246064: Deleting consensus metadata
I20260808 02:17:02.132046 7451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:02.131595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:40045 (local address 127.0.0.1:47822)
0808 02:17:02.131861 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:02.131942 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:40045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260808 02:17:02.132277 6910 catalog_manager.cc:4584] TS b0076b74a4304258a32008d5499399c4 (127.18.184.129:40045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 967e698772db41e19406e1889afc9c2e: Network error: Client connection negotiation failed: client connection to 127.18.184.129:40045: connect: Connection refused (error 111)
I20260808 02:17:02.132606 6909 catalog_manager.cc:4841] TS 2f4b1f6065b94876b9186d01db246064 (127.18.184.131:35261): tablet 967e698772db41e19406e1889afc9c2e (table test-table [id=92632d28e0ea40dbb153cd975f203645]) successfully deleted
I20260808 02:17:02.137997 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 7289
I20260808 02:17:02.140384 7451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:02.138217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:38655 (local address 127.0.0.1:58718)
0808 02:17:02.140163 (+ 1946us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:02.140253 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.130:38655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1804}
I20260808 02:17:02.146955 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 6887
2026-08-08T02:17:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1413 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:02 dist-test-slave-9gcw krb5kdc[7485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:02 dist-test-slave-9gcw krb5kdc[7485](info): set up 2 sockets
Aug 08 02:17:02 dist-test-slave-9gcw krb5kdc[7485](info): commencing operation
krb5kdc: starting...
W20260808 02:17:03.300573 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:03 dist-test-slave-9gcw krb5kdc[7485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:17:03.438812 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:35437
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:35437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:03.556794 7498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:03.557200 7498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:03.557296 7498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:03.561376 7498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:03.561491 7498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:03.561532 7498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:03.561570 7498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:03.561606 7498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:03.567011 7498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:35437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:03.568176 7498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:03.569314 7498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:03.578121 7509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:03.578121 7506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:03.578130 7504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:03.578383 7498 server_base.cc:1034] running on GCE node
I20260808 02:17:03.579074 7498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:03.580519 7498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:03.581723 7498 hybrid_clock.cc:648] HybridClock initialized: now 1786155423581708 us; error 28 us; skew 500 ppm
Aug 08 02:17:03 dist-test-slave-9gcw krb5kdc[7485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155423, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:03.584761 7498 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:17:03.585917 7498 webserver.cc:460] Webserver started at http://127.18.184.190:38147/ using document root <none> and password file <none>
I20260808 02:17:03.586575 7498 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:03.586676 7498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:03.586961 7498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:03.589332 7498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "dcf4e6b5651c4e8d873abb6822034b36"
format_stamp: "Formatted at 2026-08-08 02:17:03 on dist-test-slave-9gcw"
server_key: "8f0e8e8cbef79276e4351976f6ce2989"
server_key_iv: "cfa2a3ea4ccfec002991bc18f8d06254"
server_key_version: "encryptionkey@0"
I20260808 02:17:03.590023 7498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "dcf4e6b5651c4e8d873abb6822034b36"
format_stamp: "Formatted at 2026-08-08 02:17:03 on dist-test-slave-9gcw"
server_key: "8f0e8e8cbef79276e4351976f6ce2989"
server_key_iv: "cfa2a3ea4ccfec002991bc18f8d06254"
server_key_version: "encryptionkey@0"
I20260808 02:17:03.594421 7498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:17:03.597362 7518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:03.598773 7498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:17:03.598963 7498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "dcf4e6b5651c4e8d873abb6822034b36"
format_stamp: "Formatted at 2026-08-08 02:17:03 on dist-test-slave-9gcw"
server_key: "8f0e8e8cbef79276e4351976f6ce2989"
server_key_iv: "cfa2a3ea4ccfec002991bc18f8d06254"
server_key_version: "encryptionkey@0"
I20260808 02:17:03.599095 7498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:03.608983 7498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:03.612452 7498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:03.612687 7498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:03.620414 7498 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:35437
I20260808 02:17:03.620441 7579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:35437 every 8 connection(s)
I20260808 02:17:03.621578 7498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:03.624828 7580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:03.626642 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 7498
I20260808 02:17:03.626788 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:03.627174 19170 external_mini_cluster.cc:1428] Setting key a524a4a694ddb85cce1f335cdce403a3
I20260808 02:17:03.631963 7580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36: Bootstrap starting.
I20260808 02:17:03.634478 7580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:03.635490 7580 log.cc:824] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:17:03 dist-test-slave-9gcw krb5kdc[7485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:17:03.638077 7580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36: No bootstrap required, opened a new log
I20260808 02:17:03.642383 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:03.628876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60166 (local address 127.18.184.190:35437)
0808 02:17:03.629386 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:03.629402 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:03.629440 (+ 38us) server_negotiation.cc:407] Connection header received
0808 02:17:03.630091 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:03.630109 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:03.630452 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:03.630815 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:03.631683 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:03.632755 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:03.633729 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:03.634215 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:03.634407 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:03.637497 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:03.637502 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:03.637516 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:03.637543 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:03.639407 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:03.639900 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:03.639905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:03.639910 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:03.640023 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:03.640327 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:03.640332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:03.640334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:03.640744 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:03.640963 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:03.641320 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 02:17:03.641624 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":165,"threads_started":1}
I20260808 02:17:03.642302 7580 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf4e6b5651c4e8d873abb6822034b36" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35437 } }
I20260808 02:17:03.642562 7580 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:03.642624 7580 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf4e6b5651c4e8d873abb6822034b36, State: Initialized, Role: FOLLOWER
I20260808 02:17:03.643085 7580 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf4e6b5651c4e8d873abb6822034b36" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35437 } }
I20260808 02:17:03.643214 7580 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:03.643313 7580 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:03.643428 7580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:03.644447 7580 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf4e6b5651c4e8d873abb6822034b36" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35437 } }
I20260808 02:17:03.644843 7580 leader_election.cc:302] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcf4e6b5651c4e8d873abb6822034b36; no voters:
I20260808 02:17:03.645241 7580 leader_election.cc:288] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:03.645340 7586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:03.645617 7586 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [term 1 LEADER]: Becoming Leader. State: Replica: dcf4e6b5651c4e8d873abb6822034b36, State: Running, Role: LEADER
I20260808 02:17:03.645936 7586 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf4e6b5651c4e8d873abb6822034b36" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35437 } }
I20260808 02:17:03.646622 7580 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:03.647765 7588 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcf4e6b5651c4e8d873abb6822034b36. Latest consensus state: current_term: 1 leader_uuid: "dcf4e6b5651c4e8d873abb6822034b36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf4e6b5651c4e8d873abb6822034b36" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35437 } } }
I20260808 02:17:03.647934 7588 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:03.648226 7587 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcf4e6b5651c4e8d873abb6822034b36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf4e6b5651c4e8d873abb6822034b36" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35437 } } }
I20260808 02:17:03.648341 7587 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:03.648653 7595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:03.651608 7595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:03.658118 7595 catalog_manager.cc:1347] Generated new cluster ID: f1c3b9103ed94513836ee8a992c7c35d
I20260808 02:17:03.658202 7595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:03.681107 7595 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:03.682485 7595 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:03.693552 7595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36: Generated new TSK 0
I20260808 02:17:03.694464 7595 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.184.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
Entry for principal kudu/127.18.184.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab.
I20260808 02:17:03.729012 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:35437
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:03.846918 7607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:03.847265 7607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:03.847317 7607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:03.851678 7607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:03.851753 7607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:03.851856 7607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:17:03.857409 7607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.129.keytab
--principal=kudu/127.18.184.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:03.858577 7607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:03.859654 7607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:03.870342 7614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:03.870384 7616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:03.870354 7612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:03.870739 7607 server_base.cc:1034] running on GCE node
I20260808 02:17:03.871332 7607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:03.871963 7607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:03.873198 7607 hybrid_clock.cc:648] HybridClock initialized: now 1786155423873161 us; error 56 us; skew 500 ppm
Aug 08 02:17:03 dist-test-slave-9gcw krb5kdc[7485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155423, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:03.876303 7607 init.cc:377] Logged in from keytab as kudu/127.18.184.129@KRBTEST.COM (short username kudu)
I20260808 02:17:03.877322 7607 webserver.cc:460] Webserver started at http://127.18.184.129:38217/ using document root <none> and password file <none>
I20260808 02:17:03.877882 7607 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:03.877941 7607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:03.878114 7607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:03.880208 7607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "fa7f7bfd54804fff8687dd281e00b3ee"
format_stamp: "Formatted at 2026-08-08 02:17:03 on dist-test-slave-9gcw"
server_key: "b735f4a22fe53fa2069cdc4c3983cabd"
server_key_iv: "520895233ded934e966b1316ee3e6e3c"
server_key_version: "encryptionkey@0"
I20260808 02:17:03.880723 7607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "fa7f7bfd54804fff8687dd281e00b3ee"
format_stamp: "Formatted at 2026-08-08 02:17:03 on dist-test-slave-9gcw"
server_key: "b735f4a22fe53fa2069cdc4c3983cabd"
server_key_iv: "520895233ded934e966b1316ee3e6e3c"
server_key_version: "encryptionkey@0"
I20260808 02:17:03.884529 7607 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:17:03.887212 7623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:03.888659 7607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:17:03.888780 7607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "fa7f7bfd54804fff8687dd281e00b3ee"
format_stamp: "Formatted at 2026-08-08 02:17:03 on dist-test-slave-9gcw"
server_key: "b735f4a22fe53fa2069cdc4c3983cabd"
server_key_iv: "520895233ded934e966b1316ee3e6e3c"
server_key_version: "encryptionkey@0"
I20260808 02:17:03.888883 7607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:03.905447 7607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:03.909121 7607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:03.909399 7607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:03.910024 7607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:03.910909 7607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:03.910974 7607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:03.911018 7607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:03.911044 7607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:03.921627 7607 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:37955
I20260808 02:17:03.921649 7754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:37955 every 8 connection(s)
I20260808 02:17:03.922638 7607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:17:03.927114 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 7607
I20260808 02:17:03.927270 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:17:03.927591 19170 external_mini_cluster.cc:1428] Setting key 9d1fde8805cf15882cb6f66613a9e097
Aug 08 02:17:03 dist-test-slave-9gcw krb5kdc[7485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155423, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.129@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:17:03.938584 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:03.924635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:59145 (local address 127.18.184.190:35437)
0808 02:17:03.924847 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:03.924852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:03.925749 (+ 897us) server_negotiation.cc:407] Connection header received
0808 02:17:03.927094 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:03.927098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:03.927164 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:03.927273 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:03.928743 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:03.929520 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:03.930816 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:03.931233 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:03.931376 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:03.934576 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:03.934583 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:03.934586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:03.934620 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:03.936117 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:03.936771 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:03.936773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:03.936775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:03.936862 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:03.937333 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:03.937336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:03.937337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:03.937523 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:03.937602 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:03.938336 (+ 734us) server_negotiation.cc:299] Negotiation successful
0808 02:17:03.938453 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
WARNING: no policy specified for kudu/127.18.184.130@KRBTEST.COM; defaulting to no policy
I20260808 02:17:03.939716 7758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:03.925100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:35437 (local address 127.18.184.129:59145)
0808 02:17:03.925584 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:03.925629 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:03.926873 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:03.927472 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:03.927478 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:03.927913 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:03.928593 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:03.928614 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:03.929741 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:03.929744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:03.930674 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:03.930685 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:03.931436 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:03.931441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:03.931675 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:03.932380 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:03.932405 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:03.934366 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:03.936242 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:03.936249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:03.936270 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:03.936660 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:03.936968 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:03.936972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:03.936974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:03.937219 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:03.937634 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:03.937643 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:03.938021 (+ 378us) client_negotiation.cc:769] Sending connection context
0808 02:17:03.938262 (+ 241us) client_negotiation.cc:240] Negotiation successful
0808 02:17:03.938671 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":125,"threads_started":1}
I20260808 02:17:03.940948 7756 heartbeater.cc:347] Connected to a master server at 127.18.184.190:35437
I20260808 02:17:03.941253 7756 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:03.941774 7756 heartbeater.cc:511] Master 127.18.184.190:35437 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.130@KRBTEST.COM" created.
I20260808 02:17:03.943328 7538 ts_manager.cc:194] Registered new tserver with Master: fa7f7bfd54804fff8687dd281e00b3ee (127.18.184.129:37955)
I20260808 02:17:03.944476 7538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.129@KRBTEST.COM'} at 127.18.184.129:59145
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
Entry for principal kudu/127.18.184.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab.
I20260808 02:17:03.960244 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:35437
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:04.078416 7761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:04.078742 7761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:04.078814 7761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:04.082893 7761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:04.083009 7761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:04.083113 7761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:17:04.089510 7761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.130.keytab
--principal=kudu/127.18.184.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:04.090739 7761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:04.091843 7761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:04.102138 7767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:04.102186 7768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:04.102151 7770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:04.102552 7761 server_base.cc:1034] running on GCE node
I20260808 02:17:04.103014 7761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:04.103713 7761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:04.104911 7761 hybrid_clock.cc:648] HybridClock initialized: now 1786155424104888 us; error 40 us; skew 500 ppm
Aug 08 02:17:04 dist-test-slave-9gcw krb5kdc[7485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155424, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:04.107995 7761 init.cc:377] Logged in from keytab as kudu/127.18.184.130@KRBTEST.COM (short username kudu)
I20260808 02:17:04.108946 7761 webserver.cc:460] Webserver started at http://127.18.184.130:33581/ using document root <none> and password file <none>
I20260808 02:17:04.109658 7761 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:04.109755 7761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:04.110013 7761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:04.112154 7761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3"
format_stamp: "Formatted at 2026-08-08 02:17:04 on dist-test-slave-9gcw"
server_key: "6b074fb4bb9017141584c0724ef25e73"
server_key_iv: "923bd06c05aeb4e8abedd78dbba451a9"
server_key_version: "encryptionkey@0"
I20260808 02:17:04.112742 7761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3"
format_stamp: "Formatted at 2026-08-08 02:17:04 on dist-test-slave-9gcw"
server_key: "6b074fb4bb9017141584c0724ef25e73"
server_key_iv: "923bd06c05aeb4e8abedd78dbba451a9"
server_key_version: "encryptionkey@0"
I20260808 02:17:04.116878 7761 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 02:17:04.119686 7777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:04.121415 7761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:04.121585 7761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3"
format_stamp: "Formatted at 2026-08-08 02:17:04 on dist-test-slave-9gcw"
server_key: "6b074fb4bb9017141584c0724ef25e73"
server_key_iv: "923bd06c05aeb4e8abedd78dbba451a9"
server_key_version: "encryptionkey@0"
I20260808 02:17:04.121797 7761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:04.145569 7761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:04.149664 7761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:04.149894 7761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:04.150540 7761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:04.151818 7761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:04.151904 7761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:04.151973 7761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:04.151997 7761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:04.161586 7761 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:38705
I20260808 02:17:04.161612 7905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:38705 every 8 connection(s)
I20260808 02:17:04.162590 7761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:17:04.168639 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 7761
I20260808 02:17:04.168771 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:17:04.169103 19170 external_mini_cluster.cc:1428] Setting key 412d659e91ba3d3e3faeea5864d87459
Aug 08 02:17:04 dist-test-slave-9gcw krb5kdc[7485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155424, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.130@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:17:04.178574 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.164539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:47909 (local address 127.18.184.190:35437)
0808 02:17:04.164704 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:04.164710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:04.165671 (+ 961us) server_negotiation.cc:407] Connection header received
0808 02:17:04.167001 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:04.167004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:04.167059 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:04.167158 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:04.168426 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.169271 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.170649 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.170902 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.170987 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.174187 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:04.174190 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:04.174193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:04.174219 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:04.176052 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:04.176692 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:04.176696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:04.176699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:04.176804 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:04.177211 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:04.177218 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:04.177222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:04.177532 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:04.177657 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:04.178201 (+ 544us) server_negotiation.cc:299] Negotiation successful
0808 02:17:04.178369 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:17:04.179438 7909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.164991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:35437 (local address 127.18.184.130:47909)
0808 02:17:04.165512 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:04.165557 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:04.166801 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:04.167330 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:04.167337 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:04.167692 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:04.168200 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.168214 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.169396 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.169398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.170545 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.170555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.171065 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.171072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.171316 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.172048 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:04.172075 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:04.173995 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:04.176188 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:04.176193 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:04.176207 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:04.176578 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:04.176909 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:04.176911 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:04.176912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:04.177052 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:04.177649 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:04.177654 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:04.177930 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 02:17:04.178100 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 02:17:04.178394 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":176,"threads_started":1}
WARNING: no policy specified for kudu/127.18.184.131@KRBTEST.COM; defaulting to no policy
I20260808 02:17:04.180840 7906 heartbeater.cc:347] Connected to a master server at 127.18.184.190:35437
I20260808 02:17:04.181105 7906 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:04.181653 7906 heartbeater.cc:511] Master 127.18.184.190:35437 requested a full tablet report, sending...
I20260808 02:17:04.182806 7538 ts_manager.cc:194] Registered new tserver with Master: 5f35c4a76abe4c139f3bd07ccfdd1ac3 (127.18.184.130:38705)
I20260808 02:17:04.183465 7538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.130@KRBTEST.COM'} at 127.18.184.130:47909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
Entry for principal kudu/127.18.184.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab.
I20260808 02:17:04.201014 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:35437
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:04.319844 7912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:04.320139 7912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:04.320185 7912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:04.324262 7912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:04.324344 7912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:04.324405 7912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:17:04.330068 7912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.131.keytab
--principal=kudu/127.18.184.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:04.331214 7912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:04.332240 7912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:04.342530 7919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:04.342545 7922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:04.342549 7918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:04.343036 7912 server_base.cc:1034] running on GCE node
I20260808 02:17:04.343529 7912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:04.344251 7912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:04.345439 7912 hybrid_clock.cc:648] HybridClock initialized: now 1786155424345418 us; error 31 us; skew 500 ppm
Aug 08 02:17:04 dist-test-slave-9gcw krb5kdc[7485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155424, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:04.349084 7912 init.cc:377] Logged in from keytab as kudu/127.18.184.131@KRBTEST.COM (short username kudu)
I20260808 02:17:04.350088 7912 webserver.cc:460] Webserver started at http://127.18.184.131:36093/ using document root <none> and password file <none>
I20260808 02:17:04.350675 7912 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:04.350744 7912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:04.350946 7912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:04.353313 7912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "2d52271e48d24328946acf330c54c90e"
format_stamp: "Formatted at 2026-08-08 02:17:04 on dist-test-slave-9gcw"
server_key: "9128d88c094e0ec0bec67360a9f2836a"
server_key_iv: "f2dcdab7459e4fe7795e35dd2e42799b"
server_key_version: "encryptionkey@0"
I20260808 02:17:04.353904 7912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "2d52271e48d24328946acf330c54c90e"
format_stamp: "Formatted at 2026-08-08 02:17:04 on dist-test-slave-9gcw"
server_key: "9128d88c094e0ec0bec67360a9f2836a"
server_key_iv: "f2dcdab7459e4fe7795e35dd2e42799b"
server_key_version: "encryptionkey@0"
I20260808 02:17:04.358012 7912 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:17:04.360555 7929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:04.361941 7912 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:17:04.362048 7912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "2d52271e48d24328946acf330c54c90e"
format_stamp: "Formatted at 2026-08-08 02:17:04 on dist-test-slave-9gcw"
server_key: "9128d88c094e0ec0bec67360a9f2836a"
server_key_iv: "f2dcdab7459e4fe7795e35dd2e42799b"
server_key_version: "encryptionkey@0"
I20260808 02:17:04.362146 7912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:04.392822 7912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:04.396844 7912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:04.397027 7912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:04.397745 7912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:04.398725 7912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:04.398780 7912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:04.398826 7912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:04.398849 7912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:04.409240 7912 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:36561
I20260808 02:17:04.409261 8049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:36561 every 8 connection(s)
I20260808 02:17:04.410326 7912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:17:04.419659 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 7912
I20260808 02:17:04.419826 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:17:04.420251 19170 external_mini_cluster.cc:1428] Setting key bb02f2a6236424ea94ec594a83d8a940
Aug 08 02:17:04 dist-test-slave-9gcw krb5kdc[7485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155424, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.131@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:17:04.425592 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.412269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:36505 (local address 127.18.184.190:35437)
0808 02:17:04.412423 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:04.412435 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:04.413423 (+ 988us) server_negotiation.cc:407] Connection header received
0808 02:17:04.414876 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:04.414880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:04.414944 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:04.415028 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:04.416301 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.416856 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.418077 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.418311 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.418423 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.421684 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:04.421688 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:04.421690 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:04.421717 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:04.423227 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:04.423855 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:04.423858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:04.423859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:04.423953 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:04.424312 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:04.424316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:04.424317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:04.424521 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:04.424606 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:04.425211 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 02:17:04.425396 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:17:04.426414 8052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.412700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:35437 (local address 127.18.184.131:36505)
0808 02:17:04.413244 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:04.413294 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:04.414628 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:04.415206 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:04.415213 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:04.415615 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:04.416132 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.416147 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.416997 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.417001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.417938 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.417948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.418482 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.418486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.418656 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.419306 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:04.419345 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:04.421503 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:04.423352 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:04.423358 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:04.423383 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:04.423720 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:04.424039 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:04.424041 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:04.424042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:04.424183 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:04.424616 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:04.424621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:04.424881 (+ 260us) client_negotiation.cc:769] Sending connection context
0808 02:17:04.425035 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 02:17:04.425449 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":127,"threads_started":1}
I20260808 02:17:04.427713 8050 heartbeater.cc:347] Connected to a master server at 127.18.184.190:35437
I20260808 02:17:04.427985 8050 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:04.428498 8050 heartbeater.cc:511] Master 127.18.184.190:35437 requested a full tablet report, sending...
I20260808 02:17:04.429661 7538 ts_manager.cc:194] Registered new tserver with Master: 2d52271e48d24328946acf330c54c90e (127.18.184.131:36561)
I20260808 02:17:04.430284 7538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.184.131@KRBTEST.COM'} at 127.18.184.131:36505
I20260808 02:17:04.434808 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:17:04.444464 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.436921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60172 (local address 127.18.184.190:35437)
0808 02:17:04.437064 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:04.437068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:04.437311 (+ 243us) server_negotiation.cc:407] Connection header received
0808 02:17:04.437414 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:04.437416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:04.437460 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:04.437520 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:04.438267 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.438885 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.439799 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.440022 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.440109 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.441672 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:04.441675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:04.441678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:04.441702 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:04.442992 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:04.443375 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:04.443378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:04.443379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:04.443455 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:04.443686 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:04.443688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:04.443689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:04.443934 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:04.444041 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:04.444241 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 02:17:04.444341 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:17:04.447530 7538 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:17:04.450978 7538 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:04.467007 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.460033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:38705 (local address 127.0.0.1:47834)
0808 02:17:04.460602 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:04.460654 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:04.460808 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:04.461835 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:04.461841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:04.461866 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:04.462030 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.462042 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.464000 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.464003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.465310 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.465318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.466415 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.466418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.466516 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.466537 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 02:17:04.466621 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 02:17:04.466817 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":75,"thread_start_us":103,"threads_started":1}
I20260808 02:17:04.467434 8069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.461217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34286 (local address 127.18.184.129:37955)
0808 02:17:04.461513 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:04.461519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:04.462385 (+ 866us) server_negotiation.cc:407] Connection header received
0808 02:17:04.462452 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:04.462456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:04.462583 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:04.462726 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:04.462973 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.463841 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.465371 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.466303 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.466424 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.466548 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:04.467168 (+ 620us) server_negotiation.cc:299] Negotiation successful
0808 02:17:04.467270 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260808 02:17:04.467443 8065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.460195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.18.184.130:38705)
0808 02:17:04.461296 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:04.461307 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:04.461352 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:17:04.461436 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:04.461442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:04.461658 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:04.461832 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:04.462174 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.463413 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.465445 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.466278 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.466424 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.466548 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:04.466670 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 02:17:04.467022 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":979,"thread_start_us":149,"threads_started":1}
I20260808 02:17:04.467063 8068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.461015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:37955 (local address 127.0.0.1:34286)
0808 02:17:04.462168 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:04.462183 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:04.462274 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:04.462705 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:04.462708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:04.462725 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:04.462849 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.462863 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.463963 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.463966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.465233 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.465243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.466635 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.466639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.466739 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.466764 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:17:04.466837 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 02:17:04.466917 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1096,"mutex_wait_us":91,"thread_start_us":129,"threads_started":1}
I20260808 02:17:04.467007 8063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.460186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:36561 (local address 127.0.0.1:37740)
0808 02:17:04.460680 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:04.460699 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:04.460816 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:04.462080 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:04.462085 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:04.462105 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:04.462210 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.462220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.463564 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.463567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.465193 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.465205 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.466173 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.466176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.466312 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.466382 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 02:17:04.466587 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 02:17:04.466817 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":176,"thread_start_us":139,"threads_started":1}
I20260808 02:17:04.467674 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.460912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37740 (local address 127.18.184.131:36561)
0808 02:17:04.461626 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:04.461637 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:04.461666 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:17:04.461726 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:04.461730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:04.461940 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:04.462099 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:04.462311 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.463423 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.465325 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.465992 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.466137 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.466284 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:04.467161 (+ 877us) server_negotiation.cc:299] Negotiation successful
0808 02:17:04.467515 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":194,"threads_started":1}
I20260808 02:17:04.470467 7831 tablet_service.cc:1467] Processing CreateTablet for tablet e0fa0c6f9ddc4c79af57f95a3dc2951e (DEFAULT_TABLE table=test-table [id=0e2adb276b9240319afbb83cdc63910d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:04.470593 7676 tablet_service.cc:1467] Processing CreateTablet for tablet e0fa0c6f9ddc4c79af57f95a3dc2951e (DEFAULT_TABLE table=test-table [id=0e2adb276b9240319afbb83cdc63910d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:04.470609 7979 tablet_service.cc:1467] Processing CreateTablet for tablet e0fa0c6f9ddc4c79af57f95a3dc2951e (DEFAULT_TABLE table=test-table [id=0e2adb276b9240319afbb83cdc63910d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:04.471681 7676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0fa0c6f9ddc4c79af57f95a3dc2951e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:04.471681 7979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0fa0c6f9ddc4c79af57f95a3dc2951e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:04.471679 7831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0fa0c6f9ddc4c79af57f95a3dc2951e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:04.478623 8070 tablet_bootstrap.cc:492] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3: Bootstrap starting.
I20260808 02:17:04.480043 8072 tablet_bootstrap.cc:492] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e: Bootstrap starting.
I20260808 02:17:04.480962 8070 tablet_bootstrap.cc:654] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:04.481536 8071 tablet_bootstrap.cc:492] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee: Bootstrap starting.
I20260808 02:17:04.481952 8072 tablet_bootstrap.cc:654] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:04.482465 8070 log.cc:824] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:04.483199 8072 log.cc:824] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:04.483685 8071 tablet_bootstrap.cc:654] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:04.484386 8071 log.cc:824] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:04.485343 8072 tablet_bootstrap.cc:492] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e: No bootstrap required, opened a new log
I20260808 02:17:04.485337 8070 tablet_bootstrap.cc:492] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3: No bootstrap required, opened a new log
I20260808 02:17:04.485549 8072 ts_tablet_manager.cc:1397] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:17:04.485548 8070 ts_tablet_manager.cc:1397] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:17:04.486323 8071 tablet_bootstrap.cc:492] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee: No bootstrap required, opened a new log
I20260808 02:17:04.486536 8071 ts_tablet_manager.cc:1397] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 02:17:04.489768 8072 raft_consensus.cc:348] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38705 } } peers { permanent_uuid: "2d52271e48d24328946acf330c54c90e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36561 } } peers { permanent_uuid: "fa7f7bfd54804fff8687dd281e00b3ee" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 37955 } }
I20260808 02:17:04.490088 8072 raft_consensus.cc:374] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:04.490157 8072 raft_consensus.cc:729] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d52271e48d24328946acf330c54c90e, State: Initialized, Role: FOLLOWER
I20260808 02:17:04.490125 8071 raft_consensus.cc:348] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38705 } } peers { permanent_uuid: "2d52271e48d24328946acf330c54c90e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36561 } } peers { permanent_uuid: "fa7f7bfd54804fff8687dd281e00b3ee" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 37955 } }
I20260808 02:17:04.490427 8071 raft_consensus.cc:374] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:04.490497 8071 raft_consensus.cc:729] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa7f7bfd54804fff8687dd281e00b3ee, State: Initialized, Role: FOLLOWER
I20260808 02:17:04.489774 8070 raft_consensus.cc:348] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38705 } } peers { permanent_uuid: "2d52271e48d24328946acf330c54c90e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36561 } } peers { permanent_uuid: "fa7f7bfd54804fff8687dd281e00b3ee" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 37955 } }
I20260808 02:17:04.490576 8070 raft_consensus.cc:374] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:04.490638 8070 raft_consensus.cc:729] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f35c4a76abe4c139f3bd07ccfdd1ac3, State: Initialized, Role: FOLLOWER
I20260808 02:17:04.490689 8072 consensus_queue.cc:260] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38705 } } peers { permanent_uuid: "2d52271e48d24328946acf330c54c90e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36561 } } peers { permanent_uuid: "fa7f7bfd54804fff8687dd281e00b3ee" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 37955 } }
I20260808 02:17:04.491017 8071 consensus_queue.cc:260] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38705 } } peers { permanent_uuid: "2d52271e48d24328946acf330c54c90e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36561 } } peers { permanent_uuid: "fa7f7bfd54804fff8687dd281e00b3ee" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 37955 } }
I20260808 02:17:04.491057 8070 consensus_queue.cc:260] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38705 } } peers { permanent_uuid: "2d52271e48d24328946acf330c54c90e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36561 } } peers { permanent_uuid: "fa7f7bfd54804fff8687dd281e00b3ee" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 37955 } }
I20260808 02:17:04.491729 7756 heartbeater.cc:503] Master 127.18.184.190:35437 was elected leader, sending a full tablet report...
I20260808 02:17:04.491820 8072 ts_tablet_manager.cc:1428] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260808 02:17:04.491909 8070 ts_tablet_manager.cc:1428] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:17:04.492084 8050 heartbeater.cc:503] Master 127.18.184.190:35437 was elected leader, sending a full tablet report...
I20260808 02:17:04.492471 7906 heartbeater.cc:503] Master 127.18.184.190:35437 was elected leader, sending a full tablet report...
I20260808 02:17:04.493002 8071 ts_tablet_manager.cc:1428] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:17:04.551239 8078 raft_consensus.cc:482] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:04.551507 8078 raft_consensus.cc:504] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38705 } } peers { permanent_uuid: "2d52271e48d24328946acf330c54c90e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36561 } } peers { permanent_uuid: "fa7f7bfd54804fff8687dd281e00b3ee" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 37955 } }
I20260808 02:17:04.552738 8078 leader_election.cc:288] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d52271e48d24328946acf330c54c90e (127.18.184.131:36561), fa7f7bfd54804fff8687dd281e00b3ee (127.18.184.129:37955)
I20260808 02:17:04.557950 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.553206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:45763 (local address 127.18.184.131:36561)
0808 02:17:04.553343 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:04.553348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:04.553369 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:17:04.553542 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:04.553545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:04.553607 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:04.553695 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:04.554038 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.554809 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.556525 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.557203 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.557325 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.557380 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:04.557726 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 02:17:04.557822 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 02:17:04.557950 8069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.553214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:59063 (local address 127.18.184.129:37955)
0808 02:17:04.553368 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:04.553373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:04.553841 (+ 468us) server_negotiation.cc:407] Connection header received
0808 02:17:04.553930 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:04.553932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:04.553971 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:04.554071 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:04.554334 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.555017 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.556525 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.557184 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.557325 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.557380 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:04.557726 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 02:17:04.557806 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:17:04.557978 7909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.553053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:36561 (local address 127.18.184.130:45763)
0808 02:17:04.553253 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:04.553271 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:04.553398 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:04.553741 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:04.553747 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:04.553771 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:04.553919 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.553928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.554922 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.554924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.556369 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.556382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.557329 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.557333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.557539 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.557603 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 02:17:04.557689 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:17:04.557803 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":67}
I20260808 02:17:04.557986 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.553053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:37955 (local address 127.18.184.130:59063)
0808 02:17:04.553699 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:04.553714 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:04.553836 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:04.554106 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:04.554108 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:04.554124 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:04.554224 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.554229 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.555215 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.555219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.556393 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:04.556400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.557329 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.557333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:04.557543 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.557610 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 02:17:04.557689 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 02:17:04.557813 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":398,"thread_start_us":200,"threads_started":1}
I20260808 02:17:04.558779 7700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fa0c6f9ddc4c79af57f95a3dc2951e" candidate_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa7f7bfd54804fff8687dd281e00b3ee" is_pre_election: true
I20260808 02:17:04.558799 7999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fa0c6f9ddc4c79af57f95a3dc2951e" candidate_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d52271e48d24328946acf330c54c90e" is_pre_election: true
I20260808 02:17:04.559123 7700 raft_consensus.cc:2454] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f35c4a76abe4c139f3bd07ccfdd1ac3 in term 0.
I20260808 02:17:04.559124 7999 raft_consensus.cc:2454] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f35c4a76abe4c139f3bd07ccfdd1ac3 in term 0.
I20260808 02:17:04.559691 7778 leader_election.cc:302] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d52271e48d24328946acf330c54c90e, 5f35c4a76abe4c139f3bd07ccfdd1ac3, fa7f7bfd54804fff8687dd281e00b3ee; no voters:
I20260808 02:17:04.560024 8078 raft_consensus.cc:2781] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:04.560144 8078 raft_consensus.cc:482] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:04.560210 8078 raft_consensus.cc:3037] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:04.561363 8078 raft_consensus.cc:504] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38705 } } peers { permanent_uuid: "2d52271e48d24328946acf330c54c90e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36561 } } peers { permanent_uuid: "fa7f7bfd54804fff8687dd281e00b3ee" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 37955 } }
I20260808 02:17:04.561795 8078 leader_election.cc:288] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [CANDIDATE]: Term 1 election: Requested vote from peers 2d52271e48d24328946acf330c54c90e (127.18.184.131:36561), fa7f7bfd54804fff8687dd281e00b3ee (127.18.184.129:37955)
I20260808 02:17:04.562242 7999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fa0c6f9ddc4c79af57f95a3dc2951e" candidate_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d52271e48d24328946acf330c54c90e"
I20260808 02:17:04.562282 7700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fa0c6f9ddc4c79af57f95a3dc2951e" candidate_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa7f7bfd54804fff8687dd281e00b3ee"
I20260808 02:17:04.562398 7700 raft_consensus.cc:3037] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:04.562393 7999 raft_consensus.cc:3037] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:04.563814 7700 raft_consensus.cc:2454] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f35c4a76abe4c139f3bd07ccfdd1ac3 in term 1.
I20260808 02:17:04.563818 7999 raft_consensus.cc:2454] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f35c4a76abe4c139f3bd07ccfdd1ac3 in term 1.
I20260808 02:17:04.564189 7779 leader_election.cc:302] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d52271e48d24328946acf330c54c90e, 5f35c4a76abe4c139f3bd07ccfdd1ac3; no voters:
I20260808 02:17:04.564469 8078 raft_consensus.cc:2781] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:04.564723 8078 raft_consensus.cc:686] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 1 LEADER]: Becoming Leader. State: Replica: 5f35c4a76abe4c139f3bd07ccfdd1ac3, State: Running, Role: LEADER
I20260808 02:17:04.565090 8078 consensus_queue.cc:237] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38705 } } peers { permanent_uuid: "2d52271e48d24328946acf330c54c90e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36561 } } peers { permanent_uuid: "fa7f7bfd54804fff8687dd281e00b3ee" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 37955 } }
I20260808 02:17:04.568620 7535 catalog_manager.cc:5495] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f35c4a76abe4c139f3bd07ccfdd1ac3 (127.18.184.130). New cstate: current_term: 1 leader_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f35c4a76abe4c139f3bd07ccfdd1ac3" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 38705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d52271e48d24328946acf330c54c90e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa7f7bfd54804fff8687dd281e00b3ee" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 37955 } health_report { overall_health: UNKNOWN } } }
I20260808 02:17:04.593241 8065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.589406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47838 (local address 127.18.184.130:38705)
0808 02:17:04.589557 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:04.589562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:04.589579 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:17:04.589703 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:04.589706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:04.589767 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:04.589894 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:04.590162 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.590671 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.591533 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.591754 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.591869 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.592229 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:04.592735 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:04.592878 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:04.592939 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 02:17:04.593003 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:17:04.599750 7700 raft_consensus.cc:1261] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee [term 1 FOLLOWER]: Refusing update from remote peer 5f35c4a76abe4c139f3bd07ccfdd1ac3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:04.599750 7999 raft_consensus.cc:1261] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e [term 1 FOLLOWER]: Refusing update from remote peer 5f35c4a76abe4c139f3bd07ccfdd1ac3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:04.600538 8078 consensus_queue.cc:1035] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa7f7bfd54804fff8687dd281e00b3ee" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 37955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:04.600808 8080 consensus_queue.cc:1035] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d52271e48d24328946acf330c54c90e" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 36561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:04.608789 8086 mvcc.cc:205] Tried to move back new op lower bound from 7316092619152191488 to 7316092619021492224. Current Snapshot: MvccSnapshot[applied={T|T < 7316092619152191488}]
I20260808 02:17:04.609666 8083 mvcc.cc:205] Tried to move back new op lower bound from 7316092619152191488 to 7316092619021492224. Current Snapshot: MvccSnapshot[applied={T|T < 7316092619152191488}]
I20260808 02:17:04.609975 8085 mvcc.cc:205] Tried to move back new op lower bound from 7316092619152191488 to 7316092619021492224. Current Snapshot: MvccSnapshot[applied={T|T < 7316092619152191488}]
I20260808 02:17:04.619074 7538 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:04.619289 7538 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:04.622225 7538 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dcf4e6b5651c4e8d873abb6822034b36: Sending DeleteTablet for 3 replicas of tablet e0fa0c6f9ddc4c79af57f95a3dc2951e
I20260808 02:17:04.623179 7831 tablet_service.cc:1514] Processing DeleteTablet for tablet e0fa0c6f9ddc4c79af57f95a3dc2951e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:04 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:47834
I20260808 02:17:04.623437 7676 tablet_service.cc:1514] Processing DeleteTablet for tablet e0fa0c6f9ddc4c79af57f95a3dc2951e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:04 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:34286
I20260808 02:17:04.624018 8095 tablet_replica.cc:330] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee: stopping tablet replica
I20260808 02:17:04.624444 8095 raft_consensus.cc:2229] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:04.624420 7979 tablet_service.cc:1514] Processing DeleteTablet for tablet e0fa0c6f9ddc4c79af57f95a3dc2951e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:04 UTC) from {username='kudu', principal='kudu/127.18.184.190@KRBTEST.COM'} at 127.0.0.1:37740
I20260808 02:17:04.624961 8095 raft_consensus.cc:2258] T e0fa0c6f9ddc4c79af57f95a3dc2951e P fa7f7bfd54804fff8687dd281e00b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:04.626834 8096 tablet_replica.cc:330] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e: stopping tablet replica
I20260808 02:17:04.627203 8096 raft_consensus.cc:2229] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:04.627454 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 7607
I20260808 02:17:04.628624 8096 raft_consensus.cc:2258] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:04.640095 8096 ts_tablet_manager.cc:1905] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:04.641120 8094 tablet_replica.cc:330] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3: stopping tablet replica
I20260808 02:17:04.641499 8094 raft_consensus.cc:2229] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:17:04.642004 8094 raft_consensus.cc:2258] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:04.643721 8094 ts_tablet_manager.cc:1905] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:04.643972 8096 ts_tablet_manager.cc:1918] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:17:04.644098 8096 log.cc:1196] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/e0fa0c6f9ddc4c79af57f95a3dc2951e
I20260808 02:17:04.644481 8096 ts_tablet_manager.cc:1939] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 2d52271e48d24328946acf330c54c90e: Deleting consensus metadata
I20260808 02:17:04.645995 7520 catalog_manager.cc:4841] TS 2d52271e48d24328946acf330c54c90e (127.18.184.131:36561): tablet e0fa0c6f9ddc4c79af57f95a3dc2951e (table test-table [id=0e2adb276b9240319afbb83cdc63910d]) successfully deleted
I20260808 02:17:04.646555 8094 ts_tablet_manager.cc:1918] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:17:04.646646 8094 log.cc:1196] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/e0fa0c6f9ddc4c79af57f95a3dc2951e
I20260808 02:17:04.646981 8094 ts_tablet_manager.cc:1939] T e0fa0c6f9ddc4c79af57f95a3dc2951e P 5f35c4a76abe4c139f3bd07ccfdd1ac3: Deleting consensus metadata
I20260808 02:17:04.648399 7522 catalog_manager.cc:4841] TS 5f35c4a76abe4c139f3bd07ccfdd1ac3 (127.18.184.130:38705): tablet e0fa0c6f9ddc4c79af57f95a3dc2951e (table test-table [id=0e2adb276b9240319afbb83cdc63910d]) successfully deleted
W20260808 02:17:04.650444 7519 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:37955 (error 108)
I20260808 02:17:04.650590 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 7761
I20260808 02:17:04.653293 8068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.652917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:37955 (local address 127.0.0.1:34300)
0808 02:17:04.653061 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:04.653136 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:37955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 02:17:04.653497 7519 catalog_manager.cc:4584] TS fa7f7bfd54804fff8687dd281e00b3ee (127.18.184.129:37955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e0fa0c6f9ddc4c79af57f95a3dc2951e: Network error: Client connection negotiation failed: client connection to 127.18.184.129:37955: connect: Connection refused (error 111)
I20260808 02:17:04.658823 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 7912
I20260808 02:17:04.666074 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 7498
2026-08-08T02:17:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2520 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T02:17:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:04Z Disabled control of system clock
I20260808 02:17:04.702726 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:45073
--webserver_interface=127.18.184.190
--webserver_port=0
--builtin_ntp_servers=127.18.184.148:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:45073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:17:04.821885 8102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:04.822192 8102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:04.822232 8102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:04.826536 8102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:04.826618 8102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:04.826634 8102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:04.826673 8102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:04.826690 8102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:04.832526 8102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:45073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:45073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:04.833981 8102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:04.835603 8102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:04.845408 8110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:04.845391 8112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:04.845507 8102 server_base.cc:1034] running on GCE node
W20260808 02:17:04.845387 8109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:04.846346 8102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:04.847616 8102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:04.848853 8102 hybrid_clock.cc:648] HybridClock initialized: now 1786155424848823 us; error 41 us; skew 500 ppm
I20260808 02:17:04.855322 8102 webserver.cc:460] Webserver started at http://127.18.184.190:38345/ using document root <none> and password file <none>
I20260808 02:17:04.856038 8102 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:04.856151 8102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:04.856423 8102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:04.858753 8102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "63085ab231934cecb1c2778dc254c6aa"
format_stamp: "Formatted at 2026-08-08 02:17:04 on dist-test-slave-9gcw"
server_key: "e64475044c6028f6ce29923980c5eb6f"
server_key_iv: "2794e3f47a49f9f086f73b70cc07199c"
server_key_version: "encryptionkey@0"
I20260808 02:17:04.859444 8102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "63085ab231934cecb1c2778dc254c6aa"
format_stamp: "Formatted at 2026-08-08 02:17:04 on dist-test-slave-9gcw"
server_key: "e64475044c6028f6ce29923980c5eb6f"
server_key_iv: "2794e3f47a49f9f086f73b70cc07199c"
server_key_version: "encryptionkey@0"
I20260808 02:17:04.863526 8102 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:17:04.866307 8118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:04.867799 8102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:04.867964 8102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "63085ab231934cecb1c2778dc254c6aa"
format_stamp: "Formatted at 2026-08-08 02:17:04 on dist-test-slave-9gcw"
server_key: "e64475044c6028f6ce29923980c5eb6f"
server_key_iv: "2794e3f47a49f9f086f73b70cc07199c"
server_key_version: "encryptionkey@0"
I20260808 02:17:04.868095 8102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:04.896675 8102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:04.897557 8102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:04.897799 8102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:04.905735 8102 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:45073
I20260808 02:17:04.905820 8182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:45073 every 8 connection(s)
I20260808 02:17:04.906798 8102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:04.909991 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 8102
I20260808 02:17:04.910054 8183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:04.910181 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:04.910503 19170 external_mini_cluster.cc:1428] Setting key cc6e5f2e664a02dce403b813aaefc145
I20260808 02:17:04.916882 8183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa: Bootstrap starting.
I20260808 02:17:04.919176 8183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:04.920073 8183 log.cc:824] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:04.920403 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.911969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52056 (local address 127.18.184.190:45073)
0808 02:17:04.912712 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:04.912731 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:04.912770 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:17:04.913580 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:04.913601 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:04.914025 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:04.914375 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:04.914831 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.915871 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.916794 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.917401 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.917588 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.918539 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:04.918544 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:04.918557 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:04.918959 (+ 402us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:04.919020 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:04.919041 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:04.919159 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:04.919328 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 02:17:04.919601 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":153,"threads_started":1}
I20260808 02:17:04.922282 8183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa: No bootstrap required, opened a new log
I20260808 02:17:04.927372 8183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63085ab231934cecb1c2778dc254c6aa" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45073 } }
I20260808 02:17:04.927639 8183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:04.927682 8183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63085ab231934cecb1c2778dc254c6aa, State: Initialized, Role: FOLLOWER
I20260808 02:17:04.928169 8183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63085ab231934cecb1c2778dc254c6aa" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45073 } }
I20260808 02:17:04.928313 8183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:04.928364 8183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:04.928445 8183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:04.929580 8183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63085ab231934cecb1c2778dc254c6aa" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45073 } }
I20260808 02:17:04.929934 8183 leader_election.cc:302] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63085ab231934cecb1c2778dc254c6aa; no voters:
I20260808 02:17:04.930239 8183 leader_election.cc:288] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:04.931531 8183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:04.931998 8189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:04.932260 8189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [term 1 LEADER]: Becoming Leader. State: Replica: 63085ab231934cecb1c2778dc254c6aa, State: Running, Role: LEADER
I20260808 02:17:04.932688 8189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63085ab231934cecb1c2778dc254c6aa" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45073 } }
I20260808 02:17:04.934629 8188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63085ab231934cecb1c2778dc254c6aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63085ab231934cecb1c2778dc254c6aa" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45073 } } }
I20260808 02:17:04.934882 8188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:04.935233 8190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63085ab231934cecb1c2778dc254c6aa. Latest consensus state: current_term: 1 leader_uuid: "63085ab231934cecb1c2778dc254c6aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63085ab231934cecb1c2778dc254c6aa" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45073 } } }
I20260808 02:17:04.935341 8190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa [sys.catalog]: This master's current role is: LEADER
W20260808 02:17:04.939929 8203 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:04.940045 8203 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:04.940150 8202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:04.941087 8202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:04.946719 8202 catalog_manager.cc:1347] Generated new cluster ID: 9df8acb3acd54cb7b444a0d441ba41e1
I20260808 02:17:04.946816 8202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:04.960604 8202 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:04.961644 8202 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:04.973753 8202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa: Generated new TSK 0
I20260808 02:17:04.974700 8202 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:04.986929 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.129:0
--local_ip_for_outbound_sockets=127.18.184.129
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:45073
--builtin_ntp_servers=127.18.184.148:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:17:05.110565 8209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:05.110988 8209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:05.111119 8209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:05.115468 8209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:05.115563 8209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:05.115667 8209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.129
I20260808 02:17:05.120710 8209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.184.129
--webserver_port=0
--tserver_master_addrs=127.18.184.190:45073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.129
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:05.122042 8209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:05.123608 8209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:05.134357 8215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:05.134392 8214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:05.134389 8217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:05.134590 8209 server_base.cc:1034] running on GCE node
I20260808 02:17:05.135105 8209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:05.135800 8209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:05.137019 8209 hybrid_clock.cc:648] HybridClock initialized: now 1786155425136987 us; error 49 us; skew 500 ppm
I20260808 02:17:05.139132 8209 webserver.cc:460] Webserver started at http://127.18.184.129:44957/ using document root <none> and password file <none>
I20260808 02:17:05.139804 8209 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:05.139906 8209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:05.140151 8209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:05.142354 8209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/data/instance:
uuid: "7423ff4035bf4b2b874ff6ab896f9945"
format_stamp: "Formatted at 2026-08-08 02:17:05 on dist-test-slave-9gcw"
server_key: "c51a60267f3f239b1e08ec02f4e337a1"
server_key_iv: "f86a378bd6662bb60c75af228400d539"
server_key_version: "encryptionkey@0"
I20260808 02:17:05.142932 8209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance:
uuid: "7423ff4035bf4b2b874ff6ab896f9945"
format_stamp: "Formatted at 2026-08-08 02:17:05 on dist-test-slave-9gcw"
server_key: "c51a60267f3f239b1e08ec02f4e337a1"
server_key_iv: "f86a378bd6662bb60c75af228400d539"
server_key_version: "encryptionkey@0"
I20260808 02:17:05.146904 8209 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:17:05.149573 8224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:05.151080 8209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:17:05.151180 8209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/wal
uuid: "7423ff4035bf4b2b874ff6ab896f9945"
format_stamp: "Formatted at 2026-08-08 02:17:05 on dist-test-slave-9gcw"
server_key: "c51a60267f3f239b1e08ec02f4e337a1"
server_key_iv: "f86a378bd6662bb60c75af228400d539"
server_key_version: "encryptionkey@0"
I20260808 02:17:05.151316 8209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:05.199229 8209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:05.200114 8209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:05.200395 8209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:05.201068 8209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:05.202193 8209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:05.202257 8209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:05.202302 8209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:05.202320 8209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:05.212678 8209 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.129:33385
I20260808 02:17:05.212709 8352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.129:33385 every 8 connection(s)
I20260808 02:17:05.213768 8209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/data/info.pb
I20260808 02:17:05.216188 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 8209
I20260808 02:17:05.216347 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-0/wal/instance
I20260808 02:17:05.216691 19170 external_mini_cluster.cc:1428] Setting key ef304a0c551509b13422c628dec91d8b
I20260808 02:17:05.218940 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.130:0
--local_ip_for_outbound_sockets=127.18.184.130
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:45073
--builtin_ntp_servers=127.18.184.148:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:17:05.223871 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.215860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.129:58681 (local address 127.18.184.190:45073)
0808 02:17:05.216066 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:05.216072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:05.217277 (+ 1205us) server_negotiation.cc:407] Connection header received
0808 02:17:05.218761 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:05.218765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:05.218822 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:05.218905 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:05.219951 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.220497 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.221788 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.222017 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.222134 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.223329 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:05.223338 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:05.223340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:05.223353 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:05.223380 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:05.223388 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:05.223465 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:05.223645 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 02:17:05.223716 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 02:17:05.224784 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.216241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:45073 (local address 127.18.184.129:58681)
0808 02:17:05.217079 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:05.217135 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:05.218539 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:05.219115 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:05.219127 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:05.219264 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:05.219786 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.219809 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.220647 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.220655 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.221651 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.221661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.222185 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.222189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.222365 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.223034 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:17:05.223036 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:05.223080 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:17:05.223149 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:05.223483 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:05.223494 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:05.223507 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 02:17:05.223643 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 02:17:05.223954 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":152,"threads_started":1}
I20260808 02:17:05.226271 8353 heartbeater.cc:347] Connected to a master server at 127.18.184.190:45073
I20260808 02:17:05.226542 8353 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:05.227095 8353 heartbeater.cc:511] Master 127.18.184.190:45073 requested a full tablet report, sending...
I20260808 02:17:05.228978 8139 ts_manager.cc:194] Registered new tserver with Master: 7423ff4035bf4b2b874ff6ab896f9945 (127.18.184.129:33385)
I20260808 02:17:05.230973 8139 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.184.129:58681
W20260808 02:17:05.341297 8356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:05.341616 8356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:05.341657 8356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:05.345724 8356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:05.345811 8356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:05.345896 8356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.130
I20260808 02:17:05.350759 8356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.184.130
--webserver_port=0
--tserver_master_addrs=127.18.184.190:45073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.130
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:05.351923 8356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:05.353401 8356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:05.363349 8362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:05.363409 8364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:05.363356 8361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:05.363642 8356 server_base.cc:1034] running on GCE node
I20260808 02:17:05.364192 8356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:05.364853 8356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:05.366050 8356 hybrid_clock.cc:648] HybridClock initialized: now 1786155425366019 us; error 47 us; skew 500 ppm
I20260808 02:17:05.368024 8356 webserver.cc:460] Webserver started at http://127.18.184.130:40715/ using document root <none> and password file <none>
I20260808 02:17:05.368623 8356 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:05.368723 8356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:05.368976 8356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:05.371207 8356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/data/instance:
uuid: "1d41959819774d739be0438f087e87f9"
format_stamp: "Formatted at 2026-08-08 02:17:05 on dist-test-slave-9gcw"
server_key: "487367bed1fae9bc666d0f149f4137e4"
server_key_iv: "09987bc5db93d52d6442da37b54613eb"
server_key_version: "encryptionkey@0"
I20260808 02:17:05.371832 8356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance:
uuid: "1d41959819774d739be0438f087e87f9"
format_stamp: "Formatted at 2026-08-08 02:17:05 on dist-test-slave-9gcw"
server_key: "487367bed1fae9bc666d0f149f4137e4"
server_key_iv: "09987bc5db93d52d6442da37b54613eb"
server_key_version: "encryptionkey@0"
I20260808 02:17:05.375758 8356 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 02:17:05.378331 8372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:05.379603 8356 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:17:05.379745 8356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/wal
uuid: "1d41959819774d739be0438f087e87f9"
format_stamp: "Formatted at 2026-08-08 02:17:05 on dist-test-slave-9gcw"
server_key: "487367bed1fae9bc666d0f149f4137e4"
server_key_iv: "09987bc5db93d52d6442da37b54613eb"
server_key_version: "encryptionkey@0"
I20260808 02:17:05.379869 8356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:05.399827 8356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:05.400643 8356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:05.400882 8356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:05.401551 8356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:05.402544 8356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:05.402621 8356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:05.402689 8356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:05.402737 8356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:05.412034 8356 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.130:46875
I20260808 02:17:05.412081 8495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.130:46875 every 8 connection(s)
I20260808 02:17:05.413021 8356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/data/info.pb
I20260808 02:17:05.417215 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 8356
I20260808 02:17:05.417372 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/wal/instance
I20260808 02:17:05.417711 19170 external_mini_cluster.cc:1428] Setting key 62594d94fbd0c3964c47253eb56b1dce
I20260808 02:17:05.419986 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.184.131:0
--local_ip_for_outbound_sockets=127.18.184.131
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:45073
--builtin_ntp_servers=127.18.184.148:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:17:05.423899 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.415016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:56521 (local address 127.18.184.190:45073)
0808 02:17:05.415176 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:05.415180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:05.416212 (+ 1032us) server_negotiation.cc:407] Connection header received
0808 02:17:05.417811 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:05.417815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:05.417877 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:05.417982 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:05.419099 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.419759 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.421585 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.421816 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.421956 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.423236 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:05.423243 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:05.423249 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:05.423278 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:05.423306 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:05.423320 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:05.423402 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:05.423608 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 02:17:05.423685 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:17:05.424713 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.415401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:45073 (local address 127.18.184.130:56521)
0808 02:17:05.416071 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:05.416111 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:05.417586 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:05.418183 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:05.418190 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:05.418276 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:05.418908 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.418927 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.419914 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.419917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.421428 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.421444 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.422019 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.422026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.422227 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.422959 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:17:05.422961 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:05.423004 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:17:05.423072 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:05.423437 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:05.423449 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:05.423455 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:17:05.423589 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 02:17:05.423824 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":116,"threads_started":1}
I20260808 02:17:05.426142 8496 heartbeater.cc:347] Connected to a master server at 127.18.184.190:45073
I20260808 02:17:05.426401 8496 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:05.426971 8496 heartbeater.cc:511] Master 127.18.184.190:45073 requested a full tablet report, sending...
I20260808 02:17:05.428093 8139 ts_manager.cc:194] Registered new tserver with Master: 1d41959819774d739be0438f087e87f9 (127.18.184.130:46875)
I20260808 02:17:05.428705 8139 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.184.130:56521
W20260808 02:17:05.538355 8500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:05.538633 8500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:05.538672 8500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:05.542800 8500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:05.542858 8500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:05.542914 8500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.184.131
I20260808 02:17:05.547813 8500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.184.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.184.131
--webserver_port=0
--tserver_master_addrs=127.18.184.190:45073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.184.131
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:05.548972 8500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:05.550411 8500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:05.560171 8505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:05.560215 8506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:05.560369 8508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:05.560230 8500 server_base.cc:1034] running on GCE node
I20260808 02:17:05.560883 8500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:05.561555 8500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:05.562742 8500 hybrid_clock.cc:648] HybridClock initialized: now 1786155425562716 us; error 47 us; skew 500 ppm
I20260808 02:17:05.564599 8500 webserver.cc:460] Webserver started at http://127.18.184.131:33753/ using document root <none> and password file <none>
I20260808 02:17:05.565228 8500 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:05.565320 8500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:05.565549 8500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:05.567586 8500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/data/instance:
uuid: "1450f6bbf976406f8457687a2678c170"
format_stamp: "Formatted at 2026-08-08 02:17:05 on dist-test-slave-9gcw"
server_key: "656010b1b2e44170598853868999eb2a"
server_key_iv: "6c1fd79d5825072a8344951a6dc48f53"
server_key_version: "encryptionkey@0"
I20260808 02:17:05.568166 8500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance:
uuid: "1450f6bbf976406f8457687a2678c170"
format_stamp: "Formatted at 2026-08-08 02:17:05 on dist-test-slave-9gcw"
server_key: "656010b1b2e44170598853868999eb2a"
server_key_iv: "6c1fd79d5825072a8344951a6dc48f53"
server_key_version: "encryptionkey@0"
I20260808 02:17:05.572046 8500 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 02:17:05.574554 8514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:05.575758 8500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:17:05.575901 8500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/wal
uuid: "1450f6bbf976406f8457687a2678c170"
format_stamp: "Formatted at 2026-08-08 02:17:05 on dist-test-slave-9gcw"
server_key: "656010b1b2e44170598853868999eb2a"
server_key_iv: "6c1fd79d5825072a8344951a6dc48f53"
server_key_version: "encryptionkey@0"
I20260808 02:17:05.576011 8500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:05.595849 8500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:05.596626 8500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:05.596834 8500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:05.597524 8500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:05.598518 8500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:05.598590 8500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:05.598660 8500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:05.598707 8500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:05.607851 8500 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.131:42575
I20260808 02:17:05.607877 8646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.131:42575 every 8 connection(s)
I20260808 02:17:05.608860 8500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/data/info.pb
I20260808 02:17:05.618531 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 8500
I20260808 02:17:05.618819 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/wal/instance
I20260808 02:17:05.618922 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.610864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.131:59233 (local address 127.18.184.190:45073)
0808 02:17:05.611036 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:05.611047 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:05.612195 (+ 1148us) server_negotiation.cc:407] Connection header received
0808 02:17:05.613751 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:05.613754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:05.613807 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:05.613894 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:05.614852 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.615454 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.616594 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.616804 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.616882 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.618235 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:05.618243 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:05.618249 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:05.618269 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:05.618300 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:05.618317 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:05.618421 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:05.618641 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 02:17:05.618721 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:17:05.619251 19170 external_mini_cluster.cc:1428] Setting key 4f4a3a9b98ce6b5a73a279aca3b3c100
I20260808 02:17:05.620054 8649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.611267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:45073 (local address 127.18.184.131:59233)
0808 02:17:05.612020 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:05.612064 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:05.613522 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:05.614056 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:05.614063 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:05.614164 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:05.614664 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.614682 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.615576 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.615578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.616480 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.616489 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.616941 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.616944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.617085 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.617902 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:17:05.617904 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:05.617944 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:17:05.618029 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:05.618438 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:05.618447 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:05.618456 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 02:17:05.618574 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 02:17:05.618993 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":115,"threads_started":1}
I20260808 02:17:05.621357 8647 heartbeater.cc:347] Connected to a master server at 127.18.184.190:45073
I20260808 02:17:05.621608 8647 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:05.622169 8647 heartbeater.cc:511] Master 127.18.184.190:45073 requested a full tablet report, sending...
I20260808 02:17:05.623190 8139 ts_manager.cc:194] Registered new tserver with Master: 1450f6bbf976406f8457687a2678c170 (127.18.184.131:42575)
I20260808 02:17:05.623798 8139 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.184.131:59233
I20260808 02:17:05.634032 19170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:17:05.640189 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.636169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52064 (local address 127.18.184.190:45073)
0808 02:17:05.636360 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:05.636364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:05.636540 (+ 176us) server_negotiation.cc:407] Connection header received
0808 02:17:05.636645 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:05.636647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:05.636692 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:05.636778 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:05.637064 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.637656 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.638573 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.638799 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.638885 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.639738 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:05.639741 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:05.639743 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:05.639755 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:05.639772 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:05.639779 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:05.639838 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:05.639937 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 02:17:05.639985 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 02:17:05.643356 8139 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:17:05.646586 8139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:05.662379 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.656650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43530 (local address 127.18.184.129:33385)
0808 02:17:05.657230 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:05.657238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:05.657263 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:17:05.657355 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:05.657361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:05.657561 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:05.657726 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:05.658043 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.659230 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.660757 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.661572 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.661700 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.661892 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:05.662080 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 02:17:05.662188 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":65,"threads_started":1}
I20260808 02:17:05.662369 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.656225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.130:46875 (local address 127.0.0.1:56666)
0808 02:17:05.656990 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:05.657004 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:05.657091 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:05.658048 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:05.658050 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:05.658065 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:05.658204 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.658211 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.659608 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.659612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.660783 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.660795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.661989 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.661994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.662081 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.662100 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 02:17:05.662165 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:17:05.662215 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"mutex_wait_us":187,"thread_start_us":101,"threads_started":1}
I20260808 02:17:05.662369 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.655687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:33385 (local address 127.0.0.1:43530)
0808 02:17:05.656165 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:05.656224 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:05.656358 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:05.657729 (+ 1371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:05.657735 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:05.657757 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:05.657907 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.657915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.659350 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.659354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.660617 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.660628 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.661710 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.661714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.661862 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.661907 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 02:17:05.662033 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 02:17:05.662177 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":69,"thread_start_us":127,"threads_started":1}
I20260808 02:17:05.662917 8665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.657057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56666 (local address 127.18.184.130:46875)
0808 02:17:05.657529 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:05.657539 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:05.657557 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:17:05.657620 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:05.657624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:05.657779 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:05.657906 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:05.658363 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.659472 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.660970 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.661595 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.661746 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.661920 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:05.662640 (+ 720us) server_negotiation.cc:299] Negotiation successful
0808 02:17:05.662752 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":183,"threads_started":1}
I20260808 02:17:05.663767 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.655926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:42575 (local address 127.0.0.1:60042)
0808 02:17:05.656845 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:05.656858 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:05.656940 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:05.657904 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:05.657906 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:05.657921 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:05.658024 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.658030 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.659082 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.659087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.660617 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.660628 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.662412 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.662417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.663431 (+ 1014us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.663458 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:17:05.663530 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 02:17:05.663605 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":69,"thread_start_us":122,"threads_started":1}
I20260808 02:17:05.664350 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.656641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.18.184.131:42575)
0808 02:17:05.657413 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:05.657425 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:05.657451 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:17:05.657528 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:05.657535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:05.657731 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:05.657890 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:05.658139 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.658965 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.661287 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.662056 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.662177 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.662339 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:05.664051 (+ 1712us) server_negotiation.cc:299] Negotiation successful
0808 02:17:05.664156 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":165,"threads_started":1}
I20260808 02:17:05.665786 8423 tablet_service.cc:1467] Processing CreateTablet for tablet 31ef3b273cdf4b5d83636677bfb7cd6a (DEFAULT_TABLE table=test-table [id=29b7ac34262445c7a63ab77cb95036bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:05.665818 8279 tablet_service.cc:1467] Processing CreateTablet for tablet 31ef3b273cdf4b5d83636677bfb7cd6a (DEFAULT_TABLE table=test-table [id=29b7ac34262445c7a63ab77cb95036bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:05.666807 8423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ef3b273cdf4b5d83636677bfb7cd6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:05.666857 8279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ef3b273cdf4b5d83636677bfb7cd6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:05.666971 8568 tablet_service.cc:1467] Processing CreateTablet for tablet 31ef3b273cdf4b5d83636677bfb7cd6a (DEFAULT_TABLE table=test-table [id=29b7ac34262445c7a63ab77cb95036bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:05.667917 8568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ef3b273cdf4b5d83636677bfb7cd6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:05.673404 8666 tablet_bootstrap.cc:492] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9: Bootstrap starting.
I20260808 02:17:05.675199 8668 tablet_bootstrap.cc:492] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945: Bootstrap starting.
I20260808 02:17:05.675746 8666 tablet_bootstrap.cc:654] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:05.676504 8666 log.cc:824] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:05.677197 8668 tablet_bootstrap.cc:654] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:05.677204 8669 tablet_bootstrap.cc:492] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170: Bootstrap starting.
I20260808 02:17:05.678081 8668 log.cc:824] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:05.678915 8666 tablet_bootstrap.cc:492] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9: No bootstrap required, opened a new log
I20260808 02:17:05.679128 8666 ts_tablet_manager.cc:1397] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:17:05.679401 8669 tablet_bootstrap.cc:654] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:05.680104 8669 log.cc:824] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:05.680234 8668 tablet_bootstrap.cc:492] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945: No bootstrap required, opened a new log
I20260808 02:17:05.680419 8668 ts_tablet_manager.cc:1397] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 02:17:05.681856 8669 tablet_bootstrap.cc:492] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170: No bootstrap required, opened a new log
I20260808 02:17:05.682050 8669 ts_tablet_manager.cc:1397] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 02:17:05.683684 8666 raft_consensus.cc:348] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7423ff4035bf4b2b874ff6ab896f9945" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33385 } } peers { permanent_uuid: "1450f6bbf976406f8457687a2678c170" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42575 } } peers { permanent_uuid: "1d41959819774d739be0438f087e87f9" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46875 } }
I20260808 02:17:05.683969 8666 raft_consensus.cc:374] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:05.684028 8666 raft_consensus.cc:729] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d41959819774d739be0438f087e87f9, State: Initialized, Role: FOLLOWER
I20260808 02:17:05.684499 8666 consensus_queue.cc:260] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7423ff4035bf4b2b874ff6ab896f9945" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33385 } } peers { permanent_uuid: "1450f6bbf976406f8457687a2678c170" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42575 } } peers { permanent_uuid: "1d41959819774d739be0438f087e87f9" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46875 } }
I20260808 02:17:05.684616 8668 raft_consensus.cc:348] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7423ff4035bf4b2b874ff6ab896f9945" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33385 } } peers { permanent_uuid: "1450f6bbf976406f8457687a2678c170" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42575 } } peers { permanent_uuid: "1d41959819774d739be0438f087e87f9" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46875 } }
I20260808 02:17:05.684875 8668 raft_consensus.cc:374] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:05.684924 8668 raft_consensus.cc:729] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7423ff4035bf4b2b874ff6ab896f9945, State: Initialized, Role: FOLLOWER
I20260808 02:17:05.685375 8668 consensus_queue.cc:260] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7423ff4035bf4b2b874ff6ab896f9945" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33385 } } peers { permanent_uuid: "1450f6bbf976406f8457687a2678c170" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42575 } } peers { permanent_uuid: "1d41959819774d739be0438f087e87f9" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46875 } }
I20260808 02:17:05.685575 8666 ts_tablet_manager.cc:1428] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 02:17:05.686020 8496 heartbeater.cc:503] Master 127.18.184.190:45073 was elected leader, sending a full tablet report...
I20260808 02:17:05.686347 8668 ts_tablet_manager.cc:1428] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:17:05.686229 8669 raft_consensus.cc:348] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7423ff4035bf4b2b874ff6ab896f9945" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33385 } } peers { permanent_uuid: "1450f6bbf976406f8457687a2678c170" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42575 } } peers { permanent_uuid: "1d41959819774d739be0438f087e87f9" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46875 } }
I20260808 02:17:05.686450 8669 raft_consensus.cc:374] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:05.686510 8669 raft_consensus.cc:729] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1450f6bbf976406f8457687a2678c170, State: Initialized, Role: FOLLOWER
I20260808 02:17:05.686744 8353 heartbeater.cc:503] Master 127.18.184.190:45073 was elected leader, sending a full tablet report...
I20260808 02:17:05.686934 8669 consensus_queue.cc:260] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7423ff4035bf4b2b874ff6ab896f9945" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33385 } } peers { permanent_uuid: "1450f6bbf976406f8457687a2678c170" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42575 } } peers { permanent_uuid: "1d41959819774d739be0438f087e87f9" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46875 } }
I20260808 02:17:05.687582 8647 heartbeater.cc:503] Master 127.18.184.190:45073 was elected leader, sending a full tablet report...
I20260808 02:17:05.687866 8669 ts_tablet_manager.cc:1428] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 02:17:05.715139 8354 tablet.cc:2369] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:17:05.822597 8673 raft_consensus.cc:482] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:05.822829 8673 raft_consensus.cc:504] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7423ff4035bf4b2b874ff6ab896f9945" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33385 } } peers { permanent_uuid: "1450f6bbf976406f8457687a2678c170" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42575 } } peers { permanent_uuid: "1d41959819774d739be0438f087e87f9" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46875 } }
I20260808 02:17:05.824066 8673 leader_election.cc:288] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7423ff4035bf4b2b874ff6ab896f9945 (127.18.184.129:33385), 1450f6bbf976406f8457687a2678c170 (127.18.184.131:42575)
I20260808 02:17:05.828913 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.824449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:36197 (local address 127.18.184.129:33385)
0808 02:17:05.824590 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:05.824595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:05.824615 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:17:05.824711 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:05.824714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:05.824771 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:05.824867 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:05.825228 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.826009 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.827634 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.828247 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.828392 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.828465 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:05.828675 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 02:17:05.828777 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:17:05.828907 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.824289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:33385 (local address 127.18.184.130:36197)
0808 02:17:05.824468 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:05.824487 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:05.824596 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:05.824903 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:05.824906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:05.824925 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:05.825072 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.825081 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.826150 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.826153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.827484 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.827494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.828392 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.828395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.828518 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.828564 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 02:17:05.828641 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 02:17:05.828732 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":32}
I20260808 02:17:05.828913 8676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.824444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.131:42575 (local address 127.18.184.130:44527)
0808 02:17:05.824986 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:05.825000 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:05.825107 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:05.825461 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:05.825463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:05.825481 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:05.825584 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.825589 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.826470 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.826474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.827650 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:05.827657 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.828511 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.828514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:05.828642 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.828662 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:17:05.828726 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 02:17:05.828785 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":22,"thread_start_us":164,"threads_started":1}
I20260808 02:17:05.828998 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.824636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.184.130:44527 (local address 127.18.184.131:42575)
0808 02:17:05.824772 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:05.824777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:05.825100 (+ 323us) server_negotiation.cc:407] Connection header received
0808 02:17:05.825246 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:05.825249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:05.825305 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:05.825403 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:05.825707 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.826341 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.827770 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.828385 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.828501 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.828547 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:05.828760 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 02:17:05.828874 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:17:05.829735 8592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ef3b273cdf4b5d83636677bfb7cd6a" candidate_uuid: "1d41959819774d739be0438f087e87f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1450f6bbf976406f8457687a2678c170" is_pre_election: true
I20260808 02:17:05.829923 8301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ef3b273cdf4b5d83636677bfb7cd6a" candidate_uuid: "1d41959819774d739be0438f087e87f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7423ff4035bf4b2b874ff6ab896f9945" is_pre_election: true
I20260808 02:17:05.830102 8592 raft_consensus.cc:2454] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d41959819774d739be0438f087e87f9 in term 0.
I20260808 02:17:05.830241 8301 raft_consensus.cc:2454] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d41959819774d739be0438f087e87f9 in term 0.
I20260808 02:17:05.830698 8375 leader_election.cc:302] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1450f6bbf976406f8457687a2678c170, 1d41959819774d739be0438f087e87f9; no voters:
I20260808 02:17:05.831019 8673 raft_consensus.cc:2781] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:05.831108 8673 raft_consensus.cc:482] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:05.831161 8673 raft_consensus.cc:3037] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:05.832402 8673 raft_consensus.cc:504] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7423ff4035bf4b2b874ff6ab896f9945" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33385 } } peers { permanent_uuid: "1450f6bbf976406f8457687a2678c170" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42575 } } peers { permanent_uuid: "1d41959819774d739be0438f087e87f9" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46875 } }
I20260808 02:17:05.832783 8673 leader_election.cc:288] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [CANDIDATE]: Term 1 election: Requested vote from peers 7423ff4035bf4b2b874ff6ab896f9945 (127.18.184.129:33385), 1450f6bbf976406f8457687a2678c170 (127.18.184.131:42575)
I20260808 02:17:05.833194 8592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ef3b273cdf4b5d83636677bfb7cd6a" candidate_uuid: "1d41959819774d739be0438f087e87f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1450f6bbf976406f8457687a2678c170"
I20260808 02:17:05.833182 8301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ef3b273cdf4b5d83636677bfb7cd6a" candidate_uuid: "1d41959819774d739be0438f087e87f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7423ff4035bf4b2b874ff6ab896f9945"
I20260808 02:17:05.833339 8592 raft_consensus.cc:3037] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:05.833334 8301 raft_consensus.cc:3037] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:05.834664 8301 raft_consensus.cc:2454] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d41959819774d739be0438f087e87f9 in term 1.
I20260808 02:17:05.834677 8592 raft_consensus.cc:2454] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d41959819774d739be0438f087e87f9 in term 1.
I20260808 02:17:05.835001 8376 leader_election.cc:302] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1450f6bbf976406f8457687a2678c170, 1d41959819774d739be0438f087e87f9, 7423ff4035bf4b2b874ff6ab896f9945; no voters:
I20260808 02:17:05.835168 8673 raft_consensus.cc:2781] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:05.835399 8673 raft_consensus.cc:686] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 1 LEADER]: Becoming Leader. State: Replica: 1d41959819774d739be0438f087e87f9, State: Running, Role: LEADER
I20260808 02:17:05.835747 8673 consensus_queue.cc:237] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7423ff4035bf4b2b874ff6ab896f9945" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33385 } } peers { permanent_uuid: "1450f6bbf976406f8457687a2678c170" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42575 } } peers { permanent_uuid: "1d41959819774d739be0438f087e87f9" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46875 } }
I20260808 02:17:05.838999 8138 catalog_manager.cc:5495] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d41959819774d739be0438f087e87f9 (127.18.184.130). New cstate: current_term: 1 leader_uuid: "1d41959819774d739be0438f087e87f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7423ff4035bf4b2b874ff6ab896f9945" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1450f6bbf976406f8457687a2678c170" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d41959819774d739be0438f087e87f9" member_type: VOTER last_known_addr { host: "127.18.184.130" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260808 02:17:05.851295 8665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.847535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.18.184.130:46875)
0808 02:17:05.847688 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:05.847693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:05.847708 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:17:05.847854 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:05.847856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:05.847914 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:05.848033 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:05.848283 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.848802 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.849767 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:05.849987 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:05.850099 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:05.850426 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:05.850853 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:05.850992 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:05.851057 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 02:17:05.851112 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:17:05.857867 8301 raft_consensus.cc:1261] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945 [term 1 FOLLOWER]: Refusing update from remote peer 1d41959819774d739be0438f087e87f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:05.857867 8592 raft_consensus.cc:1261] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170 [term 1 FOLLOWER]: Refusing update from remote peer 1d41959819774d739be0438f087e87f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:05.858870 8677 consensus_queue.cc:1035] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7423ff4035bf4b2b874ff6ab896f9945" member_type: VOTER last_known_addr { host: "127.18.184.129" port: 33385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:05.859115 8673 consensus_queue.cc:1035] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1450f6bbf976406f8457687a2678c170" member_type: VOTER last_known_addr { host: "127.18.184.131" port: 42575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:17:05.859895 8648 tablet.cc:2369] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:17:05.866858 8680 mvcc.cc:205] Tried to move back new op lower bound from 7316092624305012736 to 7316092624225869824. Current Snapshot: MvccSnapshot[applied={T|T < 7316092624305012736}]
I20260808 02:17:05.869426 8682 mvcc.cc:205] Tried to move back new op lower bound from 7316092624305012736 to 7316092624225869824. Current Snapshot: MvccSnapshot[applied={T|T < 7316092624305012736}]
I20260808 02:17:05.870381 8681 mvcc.cc:205] Tried to move back new op lower bound from 7316092624305012736 to 7316092624225869824. Current Snapshot: MvccSnapshot[applied={T|T < 7316092624305012736}]
I20260808 02:17:05.877090 8139 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:05.877347 8139 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:05.880170 8139 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63085ab231934cecb1c2778dc254c6aa: Sending DeleteTablet for 3 replicas of tablet 31ef3b273cdf4b5d83636677bfb7cd6a
I20260808 02:17:05.881021 8423 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ef3b273cdf4b5d83636677bfb7cd6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:05 UTC) from {username='slave'} at 127.0.0.1:56666
I20260808 02:17:05.881036 8279 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ef3b273cdf4b5d83636677bfb7cd6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:05 UTC) from {username='slave'} at 127.0.0.1:43530
I20260808 02:17:05.881572 8691 tablet_replica.cc:330] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945: stopping tablet replica
I20260808 02:17:05.881579 8568 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ef3b273cdf4b5d83636677bfb7cd6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:05 UTC) from {username='slave'} at 127.0.0.1:60042
I20260808 02:17:05.881959 8691 raft_consensus.cc:2229] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:05.882156 8694 tablet_replica.cc:330] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170: stopping tablet replica
I20260808 02:17:05.882318 8691 raft_consensus.cc:2258] T 31ef3b273cdf4b5d83636677bfb7cd6a P 7423ff4035bf4b2b874ff6ab896f9945 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:05.882715 8690 tablet_replica.cc:330] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9: stopping tablet replica
I20260808 02:17:05.883044 8690 raft_consensus.cc:2229] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:17:05.883345 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 8209
I20260808 02:17:05.885533 8690 raft_consensus.cc:2258] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:05.887079 8690 ts_tablet_manager.cc:1905] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:05.889427 8694 raft_consensus.cc:2229] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:05.889511 8690 ts_tablet_manager.cc:1918] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:17:05.889621 8690 log.cc:1196] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-1/wal/wals/31ef3b273cdf4b5d83636677bfb7cd6a
I20260808 02:17:05.889961 8690 ts_tablet_manager.cc:1939] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1d41959819774d739be0438f087e87f9: Deleting consensus metadata
I20260808 02:17:05.890125 8694 raft_consensus.cc:2258] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:05.891383 8120 catalog_manager.cc:4841] TS 1d41959819774d739be0438f087e87f9 (127.18.184.130:46875): tablet 31ef3b273cdf4b5d83636677bfb7cd6a (table test-table [id=29b7ac34262445c7a63ab77cb95036bd]) successfully deleted
I20260808 02:17:05.891793 8694 ts_tablet_manager.cc:1905] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 02:17:05.894119 8123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.184.129:33385 (error 108)
I20260808 02:17:05.894400 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 8356
I20260808 02:17:05.897464 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:05.897102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.129:33385 (local address 127.0.0.1:43544)
0808 02:17:05.897264 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:05.897364 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.129:33385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 02:17:05.897637 8123 catalog_manager.cc:4584] TS 7423ff4035bf4b2b874ff6ab896f9945 (127.18.184.129:33385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31ef3b273cdf4b5d83636677bfb7cd6a: Network error: Client connection negotiation failed: client connection to 127.18.184.129:33385: connect: Connection refused (error 111)
I20260808 02:17:05.898169 8694 ts_tablet_manager.cc:1918] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:17:05.898366 8694 log.cc:1196] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170: Deleting WAL directory at /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155161841348-19170-0/minicluster-data/ts-2/wal/wals/31ef3b273cdf4b5d83636677bfb7cd6a
I20260808 02:17:05.898702 8694 ts_tablet_manager.cc:1939] T 31ef3b273cdf4b5d83636677bfb7cd6a P 1450f6bbf976406f8457687a2678c170: Deleting consensus metadata
I20260808 02:17:05.900007 8121 catalog_manager.cc:4841] TS 1450f6bbf976406f8457687a2678c170 (127.18.184.131:42575): tablet 31ef3b273cdf4b5d83636677bfb7cd6a (table test-table [id=29b7ac34262445c7a63ab77cb95036bd]) successfully deleted
I20260808 02:17:05.907678 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 8500
I20260808 02:17:05.916747 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 8102
2026-08-08T02:17:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1249 ms)
[----------] 6 tests from EncryptionPolicyTest (11469 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:05 dist-test-slave-9gcw krb5kdc[8699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:05 dist-test-slave-9gcw krb5kdc[8699](info): set up 2 sockets
Aug 08 02:17:05 dist-test-slave-9gcw krb5kdc[8699](info): commencing operation
krb5kdc: starting...
W20260808 02:17:07.031237 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:07 dist-test-slave-9gcw krb5kdc[8699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:07.168222 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34119
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:07.285563 8713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:07.285918 8713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:07.286007 8713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:07.290268 8713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:07.290373 8713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:07.290409 8713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:07.290447 8713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:07.290488 8713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:07.296073 8713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34119
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:07.297629 8713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:07.299000 8713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:07.308820 8718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:07.308913 8723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:07.308786 8719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:07.309283 8713 server_base.cc:1034] running on GCE node
I20260808 02:17:07.309890 8713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:07.311120 8713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:07.312310 8713 hybrid_clock.cc:648] HybridClock initialized: now 1786155427312288 us; error 33 us; skew 500 ppm
Aug 08 02:17:07 dist-test-slave-9gcw krb5kdc[8699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155427, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:07.315563 8713 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:07.316622 8713 webserver.cc:460] Webserver started at http://127.0.0.1:33179/ using document root <none> and password file <none>
I20260808 02:17:07.317317 8713 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:07.317420 8713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:07.317678 8713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:07.319912 8713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "4ea59d47eb29417da3748de073674308"
format_stamp: "Formatted at 2026-08-08 02:17:07 on dist-test-slave-9gcw"
server_key: "2ab47a27c8b5fcfb04e43e52cb8c8156"
server_key_iv: "5f76b1443f498a0729d4f1cd4f7ea20d"
server_key_version: "encryptionkey@0"
I20260808 02:17:07.320537 8713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "4ea59d47eb29417da3748de073674308"
format_stamp: "Formatted at 2026-08-08 02:17:07 on dist-test-slave-9gcw"
server_key: "2ab47a27c8b5fcfb04e43e52cb8c8156"
server_key_iv: "5f76b1443f498a0729d4f1cd4f7ea20d"
server_key_version: "encryptionkey@0"
I20260808 02:17:07.324656 8713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:17:07.327494 8730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:07.328962 8713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:07.329128 8713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "4ea59d47eb29417da3748de073674308"
format_stamp: "Formatted at 2026-08-08 02:17:07 on dist-test-slave-9gcw"
server_key: "2ab47a27c8b5fcfb04e43e52cb8c8156"
server_key_iv: "5f76b1443f498a0729d4f1cd4f7ea20d"
server_key_version: "encryptionkey@0"
I20260808 02:17:07.329341 8713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:07.350709 8713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:07.353912 8713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:07.354130 8713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:07.361690 8713 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34119
I20260808 02:17:07.361716 8792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34119 every 8 connection(s)
I20260808 02:17:07.362713 8713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:07.365549 8793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:07.366220 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 8713
I20260808 02:17:07.366379 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:07.366688 19170 external_mini_cluster.cc:1428] Setting key 009e500de29fd6d12ece1478e1a6ab7c
I20260808 02:17:07.371651 8793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308: Bootstrap starting.
I20260808 02:17:07.374102 8793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:07.374972 8793 log.cc:824] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:17:07 dist-test-slave-9gcw krb5kdc[8699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:17:07.377518 8793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308: No bootstrap required, opened a new log
I20260808 02:17:07.382249 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:07.368207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38573 (local address 127.0.0.1:34119)
0808 02:17:07.368719 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:07.368735 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:07.368771 (+ 36us) server_negotiation.cc:407] Connection header received
0808 02:17:07.369500 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:07.369518 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:07.369865 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:07.370206 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:07.371077 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:07.372120 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:07.373045 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:07.373619 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:07.373797 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:07.377019 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:07.377024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:07.377038 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:07.377065 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:07.378879 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:07.379563 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:07.379568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:07.379572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:07.379687 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:07.380100 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:07.380102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:07.380104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:07.380510 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:07.380673 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:07.381121 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 02:17:07.381477 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":162,"threads_started":1}
I20260808 02:17:07.382231 8793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea59d47eb29417da3748de073674308" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34119 } }
I20260808 02:17:07.382536 8793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:07.382586 8793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea59d47eb29417da3748de073674308, State: Initialized, Role: FOLLOWER
I20260808 02:17:07.383018 8793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea59d47eb29417da3748de073674308" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34119 } }
I20260808 02:17:07.383126 8793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:07.383168 8793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:07.383245 8793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:07.384474 8793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea59d47eb29417da3748de073674308" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34119 } }
I20260808 02:17:07.384837 8793 leader_election.cc:302] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ea59d47eb29417da3748de073674308; no voters:
I20260808 02:17:07.385299 8793 leader_election.cc:288] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:07.385543 8798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:07.385608 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:07.385731 8798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [term 1 LEADER]: Becoming Leader. State: Replica: 4ea59d47eb29417da3748de073674308, State: Running, Role: LEADER
I20260808 02:17:07.386027 8798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea59d47eb29417da3748de073674308" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34119 } }
I20260808 02:17:07.386688 8793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:07.387296 8798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ea59d47eb29417da3748de073674308. Latest consensus state: current_term: 1 leader_uuid: "4ea59d47eb29417da3748de073674308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea59d47eb29417da3748de073674308" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34119 } } }
I20260808 02:17:07.387444 8798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:07.387724 8801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ea59d47eb29417da3748de073674308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea59d47eb29417da3748de073674308" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34119 } } }
I20260808 02:17:07.387851 8801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308 [sys.catalog]: This master's current role is: LEADER
W20260808 02:17:07.392457 8818 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:07.392557 8818 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:07.392650 8810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:07.393555 8810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:07.399276 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:07.389637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50167 (local address 127.0.0.1:34119)
0808 02:17:07.389748 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:07.389752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:07.390027 (+ 275us) server_negotiation.cc:407] Connection header received
0808 02:17:07.390248 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:07.390251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:07.390323 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:07.390417 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:07.391183 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:07.391934 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:07.393070 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:07.393426 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:07.393539 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:07.395427 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:07.395431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:07.395434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:07.395459 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:07.397213 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:07.397782 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:07.397785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:07.397787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:07.397880 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:07.398314 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:07.398317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:07.398319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:07.398534 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:07.398697 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:07.399021 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 02:17:07.399125 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:17:07.400509 8810 catalog_manager.cc:1347] Generated new cluster ID: af4c68f2996a48309fb10319deeef8ba
I20260808 02:17:07.400570 8810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:07.417403 8810 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:07.418687 8810 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:07.423307 8810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ea59d47eb29417da3748de073674308: Generated new TSK 0
I20260808 02:17:07.423990 8810 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:07.429787 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 8713
W20260808 02:17:07.438534 8697 connection.cc:535] client connection to 127.0.0.1:34119 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34119): Connection reset by peer (error 104)
2026-08-08T02:17:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:07 dist-test-slave-9gcw krb5kdc[8825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:07 dist-test-slave-9gcw krb5kdc[8825](info): set up 2 sockets
Aug 08 02:17:07 dist-test-slave-9gcw krb5kdc[8825](info): commencing operation
krb5kdc: starting...
W20260808 02:17:08.589231 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:08 dist-test-slave-9gcw krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:08.729612 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45937
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:08.845353 8840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:08.845669 8840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:08.845753 8840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:08.849804 8840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:08.849900 8840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:08.849936 8840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:08.849990 8840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:08.850028 8840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:08.855417 8840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45937
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:08.856530 8840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:08.857681 8840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:08.866631 8848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:08.866643 8845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:08.866621 8846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:08.866963 8840 server_base.cc:1034] running on GCE node
I20260808 02:17:08.867594 8840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:08.868868 8840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:08.870067 8840 hybrid_clock.cc:648] HybridClock initialized: now 1786155428870026 us; error 57 us; skew 500 ppm
Aug 08 02:17:08 dist-test-slave-9gcw krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155428, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:08.873080 8840 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:08.874095 8840 webserver.cc:460] Webserver started at http://127.0.0.1:34201/ using document root <none> and password file <none>
I20260808 02:17:08.874670 8840 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:08.874763 8840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:08.875001 8840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:08.877095 8840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "18bd7c7e776f4e699baaa7b7b665ec58"
format_stamp: "Formatted at 2026-08-08 02:17:08 on dist-test-slave-9gcw"
server_key: "7a3b8711a2a7c8b8f9cdd52c0b3081a0"
server_key_iv: "1d76b896e2157171a3a9c1ed8873cffe"
server_key_version: "encryptionkey@0"
I20260808 02:17:08.877764 8840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "18bd7c7e776f4e699baaa7b7b665ec58"
format_stamp: "Formatted at 2026-08-08 02:17:08 on dist-test-slave-9gcw"
server_key: "7a3b8711a2a7c8b8f9cdd52c0b3081a0"
server_key_iv: "1d76b896e2157171a3a9c1ed8873cffe"
server_key_version: "encryptionkey@0"
I20260808 02:17:08.881646 8840 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:17:08.884166 8862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:08.885587 8840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:08.885730 8840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "18bd7c7e776f4e699baaa7b7b665ec58"
format_stamp: "Formatted at 2026-08-08 02:17:08 on dist-test-slave-9gcw"
server_key: "7a3b8711a2a7c8b8f9cdd52c0b3081a0"
server_key_iv: "1d76b896e2157171a3a9c1ed8873cffe"
server_key_version: "encryptionkey@0"
I20260808 02:17:08.885825 8840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:08.910912 8840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:08.914336 8840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:08.914520 8840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:08.922405 8840 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45937
I20260808 02:17:08.922425 8918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45937 every 8 connection(s)
I20260808 02:17:08.923434 8840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:08.926456 8919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:08.927675 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 8840
I20260808 02:17:08.927825 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:08.928197 19170 external_mini_cluster.cc:1428] Setting key 5011ad3b888de292d3e7ff06211aab8a
I20260808 02:17:08.933888 8919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58: Bootstrap starting.
I20260808 02:17:08.936481 8919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:08.937433 8919 log.cc:824] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:08.939715 8919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58: No bootstrap required, opened a new log
I20260808 02:17:08.939834 8923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:08.929681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60537 (local address 127.0.0.1:45937)
0808 02:17:08.930719 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:08.930741 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:08.930790 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:17:08.931713 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:08.931737 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:08.932237 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:08.932607 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:08.933169 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:08.934302 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:08.935682 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:08.936226 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:08.936377 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:08.937413 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:08.937424 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:08.937442 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:08.938170 (+ 728us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:08.938257 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:08.938296 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:08.938454 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:08.938818 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 02:17:08.938997 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":146,"threads_started":1}
I20260808 02:17:08.943255 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:08.944015 8919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd7c7e776f4e699baaa7b7b665ec58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45937 } }
I20260808 02:17:08.944300 8919 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:08.944355 8919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18bd7c7e776f4e699baaa7b7b665ec58, State: Initialized, Role: FOLLOWER
I20260808 02:17:08.944849 8919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd7c7e776f4e699baaa7b7b665ec58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45937 } }
I20260808 02:17:08.945017 8919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:08.945084 8919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:08.945211 8919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:08.946417 8919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd7c7e776f4e699baaa7b7b665ec58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45937 } }
I20260808 02:17:08.946784 8919 leader_election.cc:302] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18bd7c7e776f4e699baaa7b7b665ec58; no voters:
I20260808 02:17:08.947114 8919 leader_election.cc:288] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:08.947185 8930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:08.947382 8930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [term 1 LEADER]: Becoming Leader. State: Replica: 18bd7c7e776f4e699baaa7b7b665ec58, State: Running, Role: LEADER
I20260808 02:17:08.947686 8930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd7c7e776f4e699baaa7b7b665ec58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45937 } }
I20260808 02:17:08.948429 8919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:08.949398 8931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18bd7c7e776f4e699baaa7b7b665ec58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd7c7e776f4e699baaa7b7b665ec58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45937 } } }
I20260808 02:17:08.949625 8931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:08.950183 8939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:08.950995 8932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18bd7c7e776f4e699baaa7b7b665ec58. Latest consensus state: current_term: 1 leader_uuid: "18bd7c7e776f4e699baaa7b7b665ec58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd7c7e776f4e699baaa7b7b665ec58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45937 } } }
I20260808 02:17:08.951310 8932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:08.951506 8923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:08.945777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60115 (local address 127.0.0.1:45937)
0808 02:17:08.945902 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:08.945906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:08.946117 (+ 211us) server_negotiation.cc:407] Connection header received
0808 02:17:08.946210 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:08.946214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:08.946276 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:08.946367 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:08.946774 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:08.947650 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:08.948648 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:08.949038 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:08.949241 (+ 203us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:08.950526 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:08.950534 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:08.950540 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:08.950560 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:08.950596 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:08.950613 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:08.950753 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:08.951328 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 02:17:08.951365 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:17:08.954151 8939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:08.960402 8939 catalog_manager.cc:1347] Generated new cluster ID: bd1858dd1f214bdcab5b7c7e59be4a75
I20260808 02:17:08.960503 8939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:08.972096 8939 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:08.973520 8939 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:08.982921 8939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18bd7c7e776f4e699baaa7b7b665ec58: Generated new TSK 0
I20260808 02:17:08.983902 8939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:08.998473 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 8840
W20260808 02:17:09.006794 8823 connection.cc:535] client connection to 127.0.0.1:45937 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45937): Connection reset by peer (error 104)
2026-08-08T02:17:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1568 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:09 dist-test-slave-9gcw krb5kdc[8951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:09 dist-test-slave-9gcw krb5kdc[8951](info): set up 2 sockets
Aug 08 02:17:09 dist-test-slave-9gcw krb5kdc[8951](info): commencing operation
krb5kdc: starting...
W20260808 02:17:10.112349 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:10 dist-test-slave-9gcw krb5kdc[8951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:10.255939 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46567
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:10.374846 8965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:10.375218 8965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:10.375304 8965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:10.379513 8965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:10.379630 8965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:10.379670 8965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:10.379709 8965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:10.379745 8965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:10.385397 8965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46567
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:10.386626 8965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:10.387749 8965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:10.396911 8971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:10.396926 8972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:10.396895 8975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:10.397109 8965 server_base.cc:1034] running on GCE node
I20260808 02:17:10.397910 8965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:10.399067 8965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:10.400276 8965 hybrid_clock.cc:648] HybridClock initialized: now 1786155430400224 us; error 30 us; skew 500 ppm
Aug 08 02:17:10 dist-test-slave-9gcw krb5kdc[8951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155430, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:10.406946 8965 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:10.407975 8965 webserver.cc:460] Webserver started at http://127.0.0.1:38563/ using document root <none> and password file <none>
I20260808 02:17:10.408566 8965 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:10.408632 8965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:10.408867 8965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:10.411091 8965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "9dcb13109fa144549dd1d630ae57766a"
format_stamp: "Formatted at 2026-08-08 02:17:10 on dist-test-slave-9gcw"
server_key: "eb0a31bfeb48ea4fc6554b695d613550"
server_key_iv: "8974d8322e95919b0bdb3ba98a45addc"
server_key_version: "encryptionkey@0"
I20260808 02:17:10.411670 8965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "9dcb13109fa144549dd1d630ae57766a"
format_stamp: "Formatted at 2026-08-08 02:17:10 on dist-test-slave-9gcw"
server_key: "eb0a31bfeb48ea4fc6554b695d613550"
server_key_iv: "8974d8322e95919b0bdb3ba98a45addc"
server_key_version: "encryptionkey@0"
I20260808 02:17:10.415582 8965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:17:10.418140 8984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:10.419421 8965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 02:17:10.419567 8965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "9dcb13109fa144549dd1d630ae57766a"
format_stamp: "Formatted at 2026-08-08 02:17:10 on dist-test-slave-9gcw"
server_key: "eb0a31bfeb48ea4fc6554b695d613550"
server_key_iv: "8974d8322e95919b0bdb3ba98a45addc"
server_key_version: "encryptionkey@0"
I20260808 02:17:10.419684 8965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:10.424441 8965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:10.427892 8965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:10.428094 8965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:10.435941 8965 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46567
I20260808 02:17:10.435973 9043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46567 every 8 connection(s)
I20260808 02:17:10.436959 8965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:10.439916 9044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:10.443962 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 8965
I20260808 02:17:10.444126 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:10.444458 19170 external_mini_cluster.cc:1428] Setting key c1201b95c162c065ec7f6143774b1f7a
I20260808 02:17:10.446192 9044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a: Bootstrap starting.
I20260808 02:17:10.448567 9044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:10.449461 9044 log.cc:824] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:10.450160 9047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:10.445993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50691 (local address 127.0.0.1:46567)
0808 02:17:10.446499 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:10.446515 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:10.446550 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:17:10.447259 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:10.447275 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:10.447559 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:10.447912 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:10.448772 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:10.448776 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:10.448794 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:10.448805 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:10.448858 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:10.448878 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:10.448997 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:10.449098 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 02:17:10.449291 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":164,"threads_started":1}
I20260808 02:17:10.451579 9044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a: No bootstrap required, opened a new log
I20260808 02:17:10.452891 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:10.455636 9044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dcb13109fa144549dd1d630ae57766a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46567 } }
I20260808 02:17:10.455904 9044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:10.455967 9044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dcb13109fa144549dd1d630ae57766a, State: Initialized, Role: FOLLOWER
I20260808 02:17:10.456435 9044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dcb13109fa144549dd1d630ae57766a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46567 } }
I20260808 02:17:10.456599 9044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:10.456696 9044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:10.456820 9044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:10.457072 9047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:10.455017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54929 (local address 127.0.0.1:46567)
0808 02:17:10.455152 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:10.455157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:10.455393 (+ 236us) server_negotiation.cc:407] Connection header received
0808 02:17:10.455532 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:10.455534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:10.455572 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:10.455657 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:10.456428 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:10.456431 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:10.456434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:10.456445 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:10.456473 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:10.456491 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:10.456585 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:10.456933 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 02:17:10.456973 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:17:10.458040 9044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dcb13109fa144549dd1d630ae57766a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46567 } }
I20260808 02:17:10.458384 9044 leader_election.cc:302] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9dcb13109fa144549dd1d630ae57766a; no voters:
I20260808 02:17:10.458703 9044 leader_election.cc:288] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:10.458845 9055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:10.459084 9055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [term 1 LEADER]: Becoming Leader. State: Replica: 9dcb13109fa144549dd1d630ae57766a, State: Running, Role: LEADER
I20260808 02:17:10.459437 9055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dcb13109fa144549dd1d630ae57766a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46567 } }
I20260808 02:17:10.460037 9044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:10.461126 9056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9dcb13109fa144549dd1d630ae57766a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dcb13109fa144549dd1d630ae57766a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46567 } } }
I20260808 02:17:10.461309 9056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:10.461594 9057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dcb13109fa144549dd1d630ae57766a. Latest consensus state: current_term: 1 leader_uuid: "9dcb13109fa144549dd1d630ae57766a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dcb13109fa144549dd1d630ae57766a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46567 } } }
I20260808 02:17:10.461751 9057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a [sys.catalog]: This master's current role is: LEADER
W20260808 02:17:10.464866 9071 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:10.464941 9071 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:10.465070 9064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:10.465821 9064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:10.471835 9064 catalog_manager.cc:1347] Generated new cluster ID: 7b7afa1dc061428abee12c65eff9a5fe
I20260808 02:17:10.471920 9064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:10.485528 9064 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:10.485956 9064 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:10.494742 9064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9dcb13109fa144549dd1d630ae57766a: Generated new TSK 0
I20260808 02:17:10.495561 9064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:10.505052 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 8965
2026-08-08T02:17:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:10 dist-test-slave-9gcw krb5kdc[9077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:10 dist-test-slave-9gcw krb5kdc[9077](info): set up 2 sockets
Aug 08 02:17:10 dist-test-slave-9gcw krb5kdc[9077](info): commencing operation
krb5kdc: starting...
W20260808 02:17:11.652179 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:11 dist-test-slave-9gcw krb5kdc[9077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:11.796640 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35197
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:11.915038 9090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:11.915410 9090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:11.915530 9090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:11.919931 9090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:11.920047 9090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:11.920084 9090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:11.920153 9090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:11.920238 9090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:11.926446 9090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:11.927683 9090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:11.928807 9090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:11.938126 9098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:11.938112 9097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:11.938189 9090 server_base.cc:1034] running on GCE node
W20260808 02:17:11.938122 9100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:11.939106 9090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:11.940337 9090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:11.941530 9090 hybrid_clock.cc:648] HybridClock initialized: now 1786155431941513 us; error 31 us; skew 500 ppm
Aug 08 02:17:11 dist-test-slave-9gcw krb5kdc[9077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155431, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:11.944526 9090 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:11.945598 9090 webserver.cc:460] Webserver started at http://127.0.0.1:41443/ using document root <none> and password file <none>
I20260808 02:17:11.946185 9090 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:11.946280 9090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:11.946525 9090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:11.948619 9090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "9e308d94f99542e28552988c1684b6af"
format_stamp: "Formatted at 2026-08-08 02:17:11 on dist-test-slave-9gcw"
server_key: "70ec95ffe23a694b3b3d5e465eb74ff4"
server_key_iv: "7e020c5f7d5863a775d5ba28e0a12c30"
server_key_version: "encryptionkey@0"
I20260808 02:17:11.949319 9090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "9e308d94f99542e28552988c1684b6af"
format_stamp: "Formatted at 2026-08-08 02:17:11 on dist-test-slave-9gcw"
server_key: "70ec95ffe23a694b3b3d5e465eb74ff4"
server_key_iv: "7e020c5f7d5863a775d5ba28e0a12c30"
server_key_version: "encryptionkey@0"
I20260808 02:17:11.953351 9090 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:17:11.955870 9107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:11.957446 9090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:11.957624 9090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "9e308d94f99542e28552988c1684b6af"
format_stamp: "Formatted at 2026-08-08 02:17:11 on dist-test-slave-9gcw"
server_key: "70ec95ffe23a694b3b3d5e465eb74ff4"
server_key_iv: "7e020c5f7d5863a775d5ba28e0a12c30"
server_key_version: "encryptionkey@0"
I20260808 02:17:11.957780 9090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:11.982237 9090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:11.986148 9090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:11.986419 9090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:11.995086 9090 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35197
I20260808 02:17:11.995124 9162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35197 every 8 connection(s)
I20260808 02:17:11.996233 9090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:11.999208 9163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:12.005236 9163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af: Bootstrap starting.
I20260808 02:17:12.005602 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 9090
I20260808 02:17:12.005775 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:12.006095 19170 external_mini_cluster.cc:1428] Setting key 5ac6bfd5c81043611117746c749d65de
I20260808 02:17:12.008092 9163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:12.008898 9163 log.cc:824] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:12.010942 9163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af: No bootstrap required, opened a new log
I20260808 02:17:12.014639 9163 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e308d94f99542e28552988c1684b6af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35197 } }
I20260808 02:17:12.014911 9163 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:12.014997 9163 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e308d94f99542e28552988c1684b6af, State: Initialized, Role: FOLLOWER
I20260808 02:17:12.015528 9163 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e308d94f99542e28552988c1684b6af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35197 } }
I20260808 02:17:12.015720 9163 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:12.015787 9163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:12.015908 9163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [term 0 FOLLOWER]: Advancing to term 1
Aug 08 02:17:12 dist-test-slave-9gcw krb5kdc[9077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:17:12.017043 9163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e308d94f99542e28552988c1684b6af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35197 } }
I20260808 02:17:12.017459 9163 leader_election.cc:302] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e308d94f99542e28552988c1684b6af; no voters:
I20260808 02:17:12.017868 9163 leader_election.cc:288] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:12.018011 9169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:12.018296 9169 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [term 1 LEADER]: Becoming Leader. State: Replica: 9e308d94f99542e28552988c1684b6af, State: Running, Role: LEADER
I20260808 02:17:12.018725 9169 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e308d94f99542e28552988c1684b6af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35197 } }
I20260808 02:17:12.019214 9163 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:12.020669 9171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e308d94f99542e28552988c1684b6af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e308d94f99542e28552988c1684b6af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35197 } } }
I20260808 02:17:12.020818 9171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:12.021061 9171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e308d94f99542e28552988c1684b6af. Latest consensus state: current_term: 1 leader_uuid: "9e308d94f99542e28552988c1684b6af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e308d94f99542e28552988c1684b6af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35197 } } }
I20260808 02:17:12.021137 9171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:12.023751 9166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:12.007597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55455 (local address 127.0.0.1:35197)
0808 02:17:12.008078 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:12.008099 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:12.008137 (+ 38us) server_negotiation.cc:407] Connection header received
0808 02:17:12.008917 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:12.008936 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:12.009436 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:12.009836 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:12.010800 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:12.011962 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:12.012952 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:12.013552 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:12.013753 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:12.016805 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:12.016811 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:12.016832 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:12.016873 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:12.019350 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:12.019968 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:12.019974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:12.019980 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:12.020140 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:12.020590 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:12.020594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:12.020598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:12.021247 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:12.022151 (+ 904us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:12.022345 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 02:17:12.022680 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":160,"threads_started":1}
W20260808 02:17:12.024358 9186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:12.024473 9186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:12.024569 9183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:12.025553 9183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:12.027566 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:12.032970 9183 catalog_manager.cc:1347] Generated new cluster ID: a60d91705ab3411aa9de57905c2d24a6
I20260808 02:17:12.033061 9183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:12.040273 9166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:12.029981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54421 (local address 127.0.0.1:35197)
0808 02:17:12.030123 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:12.030131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:12.030467 (+ 336us) server_negotiation.cc:407] Connection header received
0808 02:17:12.030647 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:12.030650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:12.030706 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:12.030809 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:12.031757 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:12.032654 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:12.033947 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:12.034366 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:12.034479 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:12.036286 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:12.036290 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:12.036293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:12.036324 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:12.038212 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:12.038743 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:12.038746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:12.038748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:12.038844 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:12.039145 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:12.039148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:12.039150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:12.039386 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:12.039535 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:12.039776 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 02:17:12.039908 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:17:12.043748 9183 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:12.045022 9183 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:12.058714 9183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e308d94f99542e28552988c1684b6af: Generated new TSK 0
I20260808 02:17:12.059551 9183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:12.068929 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 9090
2026-08-08T02:17:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1563 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:12 dist-test-slave-9gcw krb5kdc[9199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:12 dist-test-slave-9gcw krb5kdc[9199](info): set up 2 sockets
Aug 08 02:17:12 dist-test-slave-9gcw krb5kdc[9199](info): commencing operation
krb5kdc: starting...
W20260808 02:17:13.216696 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:13 dist-test-slave-9gcw krb5kdc[9199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:13.358217 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42047
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:13.477510 9214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:13.477838 9214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:13.477885 9214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:13.482025 9214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:13.482105 9214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:13.482126 9214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:13.482144 9214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:13.482159 9214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:13.488119 9214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:13.489279 9214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:13.490309 9214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:13.499285 9219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:13.499315 9224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:13.499362 9214 server_base.cc:1034] running on GCE node
W20260808 02:17:13.499289 9220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:13.500149 9214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:13.501456 9214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:13.502650 9214 hybrid_clock.cc:648] HybridClock initialized: now 1786155433502607 us; error 62 us; skew 500 ppm
Aug 08 02:17:13 dist-test-slave-9gcw krb5kdc[9199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155433, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:13.505702 9214 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:13.506647 9214 webserver.cc:460] Webserver started at http://127.0.0.1:44141/ using document root <none> and password file <none>
I20260808 02:17:13.507246 9214 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:13.507309 9214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:13.507495 9214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:13.509691 9214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "ccd54682e1034cd7918c7a36d0da5137"
format_stamp: "Formatted at 2026-08-08 02:17:13 on dist-test-slave-9gcw"
server_key: "08c25316d50222e5f598ea721f773172"
server_key_iv: "0096db812935ac04c9fc192faf1ace5d"
server_key_version: "encryptionkey@0"
I20260808 02:17:13.510227 9214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "ccd54682e1034cd7918c7a36d0da5137"
format_stamp: "Formatted at 2026-08-08 02:17:13 on dist-test-slave-9gcw"
server_key: "08c25316d50222e5f598ea721f773172"
server_key_iv: "0096db812935ac04c9fc192faf1ace5d"
server_key_version: "encryptionkey@0"
I20260808 02:17:13.514382 9214 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:17:13.517683 9231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:13.519119 9214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:17:13.519368 9214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "ccd54682e1034cd7918c7a36d0da5137"
format_stamp: "Formatted at 2026-08-08 02:17:13 on dist-test-slave-9gcw"
server_key: "08c25316d50222e5f598ea721f773172"
server_key_iv: "0096db812935ac04c9fc192faf1ace5d"
server_key_version: "encryptionkey@0"
I20260808 02:17:13.519506 9214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:13.540658 9214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:13.544138 9214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:13.544396 9214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:13.552060 9214 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42047
I20260808 02:17:13.552078 9289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42047 every 8 connection(s)
I20260808 02:17:13.553082 9214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:13.556092 9290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:13.556375 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 9214
I20260808 02:17:13.556527 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:13.556861 19170 external_mini_cluster.cc:1428] Setting key 22e8793cff2808cfdfb2c058355d1b58
I20260808 02:17:13.563620 9290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137: Bootstrap starting.
I20260808 02:17:13.566355 9290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:13.566372 9293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:13.558403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34287 (local address 127.0.0.1:42047)
0808 02:17:13.558846 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:13.558862 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:13.558898 (+ 36us) server_negotiation.cc:407] Connection header received
0808 02:17:13.559580 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:13.559597 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:13.559964 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:13.560249 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:13.560625 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:13.561705 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:13.562990 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:13.563471 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:13.563587 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:13.564570 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:13.564575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:13.564588 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:13.564994 (+ 406us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:13.565049 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:13.565073 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:13.565221 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:13.565345 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 02:17:13.565582 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":148,"threads_started":1}
I20260808 02:17:13.567150 9290 log.cc:824] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:13.569522 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:13.569703 9290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137: No bootstrap required, opened a new log
I20260808 02:17:13.573563 9290 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd54682e1034cd7918c7a36d0da5137" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } }
I20260808 02:17:13.573833 9290 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:13.573921 9290 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccd54682e1034cd7918c7a36d0da5137, State: Initialized, Role: FOLLOWER
I20260808 02:17:13.574412 9290 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd54682e1034cd7918c7a36d0da5137" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } }
I20260808 02:17:13.574568 9290 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:13.574662 9290 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:13.574796 9290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:13.575631 9293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:13.571667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48227 (local address 127.0.0.1:42047)
0808 02:17:13.571796 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:13.571801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:13.572028 (+ 227us) server_negotiation.cc:407] Connection header received
0808 02:17:13.572178 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:13.572181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:13.572232 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:13.572317 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:13.572596 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:13.573113 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:13.573998 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:13.574245 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:13.574360 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:13.575199 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:13.575203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:13.575205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:13.575217 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:13.575248 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:13.575260 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:13.575339 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:13.575446 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 02:17:13.575497 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:17:13.575937 9290 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd54682e1034cd7918c7a36d0da5137" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } }
I20260808 02:17:13.576308 9290 leader_election.cc:302] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccd54682e1034cd7918c7a36d0da5137; no voters:
I20260808 02:17:13.576678 9290 leader_election.cc:288] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:13.576815 9300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:13.577090 9300 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [term 1 LEADER]: Becoming Leader. State: Replica: ccd54682e1034cd7918c7a36d0da5137, State: Running, Role: LEADER
I20260808 02:17:13.577497 9300 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd54682e1034cd7918c7a36d0da5137" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } }
I20260808 02:17:13.578053 9290 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:13.579444 9302 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccd54682e1034cd7918c7a36d0da5137. Latest consensus state: current_term: 1 leader_uuid: "ccd54682e1034cd7918c7a36d0da5137" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd54682e1034cd7918c7a36d0da5137" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } } }
I20260808 02:17:13.579473 9301 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccd54682e1034cd7918c7a36d0da5137" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd54682e1034cd7918c7a36d0da5137" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } } }
I20260808 02:17:13.579576 9302 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:13.579576 9301 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:13.580036 9310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:13.583595 9310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:13.590201 9310 catalog_manager.cc:1347] Generated new cluster ID: e9e797e6401f4d51861b5749215c5403
I20260808 02:17:13.590288 9310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:13.608204 9310 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:13.609586 9310 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:13.615804 9310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccd54682e1034cd7918c7a36d0da5137: Generated new TSK 0
I20260808 02:17:13.616583 9310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:13.624714 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 9214
2026-08-08T02:17:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1556 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:13 dist-test-slave-9gcw krb5kdc[9322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:13 dist-test-slave-9gcw krb5kdc[9322](info): set up 2 sockets
Aug 08 02:17:13 dist-test-slave-9gcw krb5kdc[9322](info): commencing operation
krb5kdc: starting...
W20260808 02:17:14.716449 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:14 dist-test-slave-9gcw krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:14.864495 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:14.984862 9338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:14.985298 9338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:14.985357 9338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:14.989995 9338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:14.990075 9338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:14.990097 9338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:14.990116 9338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:14.990134 9338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:14.996393 9338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:14.997677 9338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:14.998765 9338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:15.008292 9347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:15.008432 9338 server_base.cc:1034] running on GCE node
W20260808 02:17:15.008294 9344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:15.008294 9345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:15.009384 9338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:15.010516 9338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:15.011713 9338 hybrid_clock.cc:648] HybridClock initialized: now 1786155435011707 us; error 36 us; skew 500 ppm
Aug 08 02:17:15 dist-test-slave-9gcw krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155435, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:15.014824 9338 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:15.015829 9338 webserver.cc:460] Webserver started at http://127.0.0.1:40607/ using document root <none> and password file <none>
I20260808 02:17:15.016367 9338 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:15.016429 9338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:15.016606 9338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:15.018781 9338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "544a9a9375e8453181b749f53cef109d"
format_stamp: "Formatted at 2026-08-08 02:17:15 on dist-test-slave-9gcw"
server_key: "375a90296e8054f6fe0165ff86cde8b5"
server_key_iv: "9498569dfcff7ffc2e5e540e56577546"
server_key_version: "encryptionkey@0"
I20260808 02:17:15.019326 9338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "544a9a9375e8453181b749f53cef109d"
format_stamp: "Formatted at 2026-08-08 02:17:15 on dist-test-slave-9gcw"
server_key: "375a90296e8054f6fe0165ff86cde8b5"
server_key_iv: "9498569dfcff7ffc2e5e540e56577546"
server_key_version: "encryptionkey@0"
I20260808 02:17:15.023423 9338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:17:15.026276 9355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:15.027777 9338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:15.027985 9338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "544a9a9375e8453181b749f53cef109d"
format_stamp: "Formatted at 2026-08-08 02:17:15 on dist-test-slave-9gcw"
server_key: "375a90296e8054f6fe0165ff86cde8b5"
server_key_iv: "9498569dfcff7ffc2e5e540e56577546"
server_key_version: "encryptionkey@0"
I20260808 02:17:15.028154 9338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:15.033576 9338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:15.037544 9338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:15.037781 9338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:15.045790 9338 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40801
I20260808 02:17:15.045879 9417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40801 every 8 connection(s)
I20260808 02:17:15.046825 9338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:15.050073 9418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:15.052786 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 9338
I20260808 02:17:15.052953 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:15.053347 19170 external_mini_cluster.cc:1428] Setting key 1d70ba0344aa7edcd42b4fd5ace7c29f
I20260808 02:17:15.057021 9418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d: Bootstrap starting.
I20260808 02:17:15.058807 9421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:15.054896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57725 (local address 127.0.0.1:40801)
0808 02:17:15.055403 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:15.055419 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:15.055455 (+ 36us) server_negotiation.cc:407] Connection header received
0808 02:17:15.056101 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:15.056118 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:15.056405 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:15.056744 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:15.057619 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:15.057623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:15.057634 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:15.057645 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:15.057697 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:15.057716 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:15.057877 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:15.057949 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 02:17:15.058115 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":175,"threads_started":1}
I20260808 02:17:15.059732 9418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:15.060730 9418 log.cc:824] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:15.062093 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:15.063297 9418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d: No bootstrap required, opened a new log
I20260808 02:17:15.066352 9421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:15.064564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53947 (local address 127.0.0.1:40801)
0808 02:17:15.064702 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:15.064707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:15.064925 (+ 218us) server_negotiation.cc:407] Connection header received
0808 02:17:15.065046 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:15.065048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:15.065079 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:15.065186 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:15.065921 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:15.065923 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:15.065925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:15.065935 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:15.065957 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:15.065966 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:15.066036 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:15.066137 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 02:17:15.066169 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:17:15.068302 9418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544a9a9375e8453181b749f53cef109d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40801 } }
I20260808 02:17:15.068547 9418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:15.068581 9418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 544a9a9375e8453181b749f53cef109d, State: Initialized, Role: FOLLOWER
I20260808 02:17:15.069037 9418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544a9a9375e8453181b749f53cef109d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40801 } }
I20260808 02:17:15.069216 9418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:15.069310 9418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:15.069435 9418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:15.070696 9418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544a9a9375e8453181b749f53cef109d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40801 } }
I20260808 02:17:15.071066 9418 leader_election.cc:302] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 544a9a9375e8453181b749f53cef109d; no voters:
I20260808 02:17:15.071412 9418 leader_election.cc:288] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:15.071566 9430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:15.071813 9430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [term 1 LEADER]: Becoming Leader. State: Replica: 544a9a9375e8453181b749f53cef109d, State: Running, Role: LEADER
I20260808 02:17:15.072150 9430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544a9a9375e8453181b749f53cef109d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40801 } }
I20260808 02:17:15.072721 9418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:15.074200 9432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 544a9a9375e8453181b749f53cef109d. Latest consensus state: current_term: 1 leader_uuid: "544a9a9375e8453181b749f53cef109d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544a9a9375e8453181b749f53cef109d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40801 } } }
I20260808 02:17:15.074206 9431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "544a9a9375e8453181b749f53cef109d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544a9a9375e8453181b749f53cef109d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40801 } } }
I20260808 02:17:15.074344 9431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:15.074344 9432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:15.075160 9439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:15.078293 9439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:15.084172 9439 catalog_manager.cc:1347] Generated new cluster ID: 20f5efde09d5454e8e0781d0e55908ee
I20260808 02:17:15.084256 9439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:15.112887 9439 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:15.113248 9439 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:15.117633 9439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 544a9a9375e8453181b749f53cef109d: Generated new TSK 0
I20260808 02:17:15.118427 9439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:15.123787 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 9338
2026-08-08T02:17:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:15 dist-test-slave-9gcw krb5kdc[9451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:15 dist-test-slave-9gcw krb5kdc[9451](info): set up 2 sockets
Aug 08 02:17:15 dist-test-slave-9gcw krb5kdc[9451](info): commencing operation
krb5kdc: starting...
W20260808 02:17:16.225087 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:16 dist-test-slave-9gcw krb5kdc[9451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:16.376718 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42669
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:16.497190 9465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:16.497512 9465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:16.497560 9465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:16.501959 9465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:16.502039 9465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:16.502061 9465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:16.502135 9465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:16.502177 9465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:16.507735 9465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:16.508948 9465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:16.510080 9465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:16.519205 9473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:16.519259 9471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:16.519289 9470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:16.519624 9465 server_base.cc:1034] running on GCE node
I20260808 02:17:16.520274 9465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:16.521593 9465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:16.522800 9465 hybrid_clock.cc:648] HybridClock initialized: now 1786155436522764 us; error 58 us; skew 500 ppm
Aug 08 02:17:16 dist-test-slave-9gcw krb5kdc[9451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155436, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:16.525801 9465 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:16.526791 9465 webserver.cc:460] Webserver started at http://127.0.0.1:43017/ using document root <none> and password file <none>
I20260808 02:17:16.527436 9465 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:16.527500 9465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:16.527729 9465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:16.530012 9465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "3ec1b60a6dd547cba540907fc5117aed"
format_stamp: "Formatted at 2026-08-08 02:17:16 on dist-test-slave-9gcw"
server_key: "6a954d9acfafef3ab1efbfb8e5e23fcc"
server_key_iv: "d9e007a53d747f9a5aaf0c43d952a7e6"
server_key_version: "encryptionkey@0"
I20260808 02:17:16.530603 9465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "3ec1b60a6dd547cba540907fc5117aed"
format_stamp: "Formatted at 2026-08-08 02:17:16 on dist-test-slave-9gcw"
server_key: "6a954d9acfafef3ab1efbfb8e5e23fcc"
server_key_iv: "d9e007a53d747f9a5aaf0c43d952a7e6"
server_key_version: "encryptionkey@0"
I20260808 02:17:16.534644 9465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:17:16.537415 9481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:16.538803 9465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:16.538964 9465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "3ec1b60a6dd547cba540907fc5117aed"
format_stamp: "Formatted at 2026-08-08 02:17:16 on dist-test-slave-9gcw"
server_key: "6a954d9acfafef3ab1efbfb8e5e23fcc"
server_key_iv: "d9e007a53d747f9a5aaf0c43d952a7e6"
server_key_version: "encryptionkey@0"
I20260808 02:17:16.539099 9465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:16.561524 9465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:16.564714 9465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:16.564965 9465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:16.572476 9465 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42669
I20260808 02:17:16.572505 9541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42669 every 8 connection(s)
I20260808 02:17:16.573549 9465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:16.575099 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 9465
I20260808 02:17:16.575251 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:16.575623 19170 external_mini_cluster.cc:1428] Setting key 40bf67b0e585c5109bc59592cfc815e6
I20260808 02:17:16.576838 9542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:16.582726 9542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed: Bootstrap starting.
I20260808 02:17:16.585050 9542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed: Neither blocks nor log segments found. Creating new log.
Aug 08 02:17:16 dist-test-slave-9gcw krb5kdc[9451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:17:16.585951 9542 log.cc:824] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:16.588203 9542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed: No bootstrap required, opened a new log
I20260808 02:17:16.591497 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.577203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:50597 (local address 127.0.0.1:42669)
0808 02:17:16.577712 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.577730 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.577830 (+ 100us) server_negotiation.cc:407] Connection header received
0808 02:17:16.578571 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.578589 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.578970 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.579344 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.580297 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.581511 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.582501 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.583097 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.583287 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.586357 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.586363 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.586379 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.586413 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.588274 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.588970 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.588975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.588979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.589092 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.589526 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.589528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.589530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.589927 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.590086 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.590461 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.590702 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":142,"threads_started":1}
I20260808 02:17:16.592953 9542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec1b60a6dd547cba540907fc5117aed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42669 } }
I20260808 02:17:16.593291 9542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:16.593395 9542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec1b60a6dd547cba540907fc5117aed, State: Initialized, Role: FOLLOWER
I20260808 02:17:16.593909 9542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec1b60a6dd547cba540907fc5117aed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42669 } }
I20260808 02:17:16.594048 9542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:16.594149 9542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:16.594285 9542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:16.594635 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:16.595700 9542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec1b60a6dd547cba540907fc5117aed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42669 } }
I20260808 02:17:16.596144 9542 leader_election.cc:302] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ec1b60a6dd547cba540907fc5117aed; no voters:
I20260808 02:17:16.596597 9542 leader_election.cc:288] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:16.596711 9552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:16.596963 9552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [term 1 LEADER]: Becoming Leader. State: Replica: 3ec1b60a6dd547cba540907fc5117aed, State: Running, Role: LEADER
I20260808 02:17:16.597357 9552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec1b60a6dd547cba540907fc5117aed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42669 } }
I20260808 02:17:16.598176 9542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:16.599128 9555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ec1b60a6dd547cba540907fc5117aed. Latest consensus state: current_term: 1 leader_uuid: "3ec1b60a6dd547cba540907fc5117aed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec1b60a6dd547cba540907fc5117aed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42669 } } }
I20260808 02:17:16.599257 9555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:16.599584 9553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ec1b60a6dd547cba540907fc5117aed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec1b60a6dd547cba540907fc5117aed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42669 } } }
I20260808 02:17:16.599637 9562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:16.599673 9553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:16.603210 9562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:16.607998 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.597760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:53409 (local address 127.0.0.1:42669)
0808 02:17:16.597889 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.597894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.597910 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:17:16.598075 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.598078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.598126 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.598212 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.599203 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.600140 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.601241 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.601639 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.601741 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.603644 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.603648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.603651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.603677 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.605537 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.606173 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.606176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.606178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.606275 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.606672 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.606675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.606677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.606896 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.607035 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.607475 (+ 440us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.607591 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:17:16.609433 9562 catalog_manager.cc:1347] Generated new cluster ID: d40d79303edd4b8e9ca11b2c50361c4e
I20260808 02:17:16.609510 9562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:16.618259 9562 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:16.619174 9562 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:16.626000 9562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ec1b60a6dd547cba540907fc5117aed: Generated new TSK 0
I20260808 02:17:16.626709 9562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:16.632817 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 9465
W20260808 02:17:16.640997 9449 connection.cc:535] client connection to 127.0.0.1:42669 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42669): Connection reset by peer (error 104)
2026-08-08T02:17:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:16 dist-test-slave-9gcw krb5kdc[9578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:16 dist-test-slave-9gcw krb5kdc[9578](info): set up 2 sockets
Aug 08 02:17:16 dist-test-slave-9gcw krb5kdc[9578](info): commencing operation
krb5kdc: starting...
W20260808 02:17:17.732251 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:17 dist-test-slave-9gcw krb5kdc[9578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:17.872810 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36495
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:17.991918 9591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:17.992269 9591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:17.992354 9591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:17.996651 9591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:17.996768 9591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:17.996803 9591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:17.996841 9591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:17.996878 9591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:18.002534 9591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:18.003748 9591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:18.004845 9591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:18.013947 9599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:18.013960 9596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:18.013947 9597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:18.014305 9591 server_base.cc:1034] running on GCE node
I20260808 02:17:18.014885 9591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:18.016070 9591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:18.017233 9591 hybrid_clock.cc:648] HybridClock initialized: now 1786155438017208 us; error 35 us; skew 500 ppm
Aug 08 02:17:18 dist-test-slave-9gcw krb5kdc[9578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:18.020316 9591 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:18.021392 9591 webserver.cc:460] Webserver started at http://127.0.0.1:34857/ using document root <none> and password file <none>
I20260808 02:17:18.021970 9591 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:18.022033 9591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:18.022256 9591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:18.024416 9591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "0641565dbc164072ae91ce98d5342ef8"
format_stamp: "Formatted at 2026-08-08 02:17:18 on dist-test-slave-9gcw"
server_key: "58424c3f7ab9de649f2b4954d290142e"
server_key_iv: "57a15bf452f63621df672f4e484adaf8"
server_key_version: "encryptionkey@0"
I20260808 02:17:18.025002 9591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "0641565dbc164072ae91ce98d5342ef8"
format_stamp: "Formatted at 2026-08-08 02:17:18 on dist-test-slave-9gcw"
server_key: "58424c3f7ab9de649f2b4954d290142e"
server_key_iv: "57a15bf452f63621df672f4e484adaf8"
server_key_version: "encryptionkey@0"
I20260808 02:17:18.028844 9591 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 02:17:18.031600 9607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:18.033084 9591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:18.033265 9591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "0641565dbc164072ae91ce98d5342ef8"
format_stamp: "Formatted at 2026-08-08 02:17:18 on dist-test-slave-9gcw"
server_key: "58424c3f7ab9de649f2b4954d290142e"
server_key_iv: "57a15bf452f63621df672f4e484adaf8"
server_key_version: "encryptionkey@0"
I20260808 02:17:18.033406 9591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:18.054849 9591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:18.058538 9591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:18.058799 9591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:18.067123 9591 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36495
I20260808 02:17:18.067154 9664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36495 every 8 connection(s)
I20260808 02:17:18.068193 9591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:18.071245 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 9591
I20260808 02:17:18.071383 9665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:18.071408 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:18.071779 19170 external_mini_cluster.cc:1428] Setting key 726866155093f44eb501637ef8ba3e04
I20260808 02:17:18.078560 9665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8: Bootstrap starting.
I20260808 02:17:18.081131 9665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:18.082079 9665 log.cc:824] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:18.082218 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.073363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:49151 (local address 127.0.0.1:36495)
0808 02:17:18.073853 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:18.073870 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:18.073910 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:17:18.074923 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:18.074943 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:18.075317 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:18.075714 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:18.076203 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.077386 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.078422 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.079012 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.079201 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:18.080295 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:18.080301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:18.080320 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:18.080731 (+ 411us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:18.080787 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:18.080823 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:18.080981 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:18.081225 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 02:17:18.081378 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":142,"threads_started":1}
I20260808 02:17:18.084615 9665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8: No bootstrap required, opened a new log
I20260808 02:17:18.085598 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:18.089483 9665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0641565dbc164072ae91ce98d5342ef8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36495 } }
I20260808 02:17:18.089779 9665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:18.089854 9665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0641565dbc164072ae91ce98d5342ef8, State: Initialized, Role: FOLLOWER
I20260808 02:17:18.090385 9665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0641565dbc164072ae91ce98d5342ef8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36495 } }
I20260808 02:17:18.090557 9665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:18.090617 9665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:18.090739 9665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:18.092090 9665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0641565dbc164072ae91ce98d5342ef8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36495 } }
I20260808 02:17:18.092525 9665 leader_election.cc:302] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0641565dbc164072ae91ce98d5342ef8; no voters:
I20260808 02:17:18.092981 9665 leader_election.cc:288] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:18.093039 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.087955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:37435 (local address 127.0.0.1:36495)
0808 02:17:18.088083 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:18.088087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:18.088342 (+ 255us) server_negotiation.cc:407] Connection header received
0808 02:17:18.088562 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:18.088565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:18.088611 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:18.088705 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:18.089043 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.089642 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.090943 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.091343 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.091460 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:18.092445 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:18.092449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:18.092453 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:18.092468 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:18.092498 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:18.092511 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:18.092615 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:18.092830 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 02:17:18.092871 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:17:18.093067 9676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:18.093698 9676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [term 1 LEADER]: Becoming Leader. State: Replica: 0641565dbc164072ae91ce98d5342ef8, State: Running, Role: LEADER
I20260808 02:17:18.094056 9676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0641565dbc164072ae91ce98d5342ef8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36495 } }
I20260808 02:17:18.094589 9665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:18.095901 9678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0641565dbc164072ae91ce98d5342ef8. Latest consensus state: current_term: 1 leader_uuid: "0641565dbc164072ae91ce98d5342ef8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0641565dbc164072ae91ce98d5342ef8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36495 } } }
I20260808 02:17:18.096050 9678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:18.096441 9677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0641565dbc164072ae91ce98d5342ef8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0641565dbc164072ae91ce98d5342ef8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36495 } } }
I20260808 02:17:18.096509 9686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:18.096571 9677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:18.099635 9686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:18.106166 9686 catalog_manager.cc:1347] Generated new cluster ID: 1130e5d14943452d91f672ff411f33f3
I20260808 02:17:18.106262 9686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:18.132671 9686 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:18.134032 9686 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:18.139083 9686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0641565dbc164072ae91ce98d5342ef8: Generated new TSK 0
I20260808 02:17:18.139889 9686 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:18.144961 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 9591
W20260808 02:17:18.152233 9575 connection.cc:535] client connection to 127.0.0.1:36495 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36495): Connection reset by peer (error 104)
2026-08-08T02:17:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1511 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:18 dist-test-slave-9gcw krb5kdc[9700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:18 dist-test-slave-9gcw krb5kdc[9700](info): set up 2 sockets
Aug 08 02:17:18 dist-test-slave-9gcw krb5kdc[9700](info): commencing operation
krb5kdc: starting...
W20260808 02:17:19.290570 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:19 dist-test-slave-9gcw krb5kdc[9700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:19.428709 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35339
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:19.546435 9715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:19.546725 9715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:19.546768 9715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:19.550910 9715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:19.550992 9715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:19.551012 9715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:19.551030 9715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:19.551046 9715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:19.556746 9715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:19.558063 9715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:19.559165 9715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:19.568241 9720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:19.568218 9721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:19.568280 9723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:19.568636 9715 server_base.cc:1034] running on GCE node
I20260808 02:17:19.569377 9715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:19.570631 9715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:19.571852 9715 hybrid_clock.cc:648] HybridClock initialized: now 1786155439571821 us; error 46 us; skew 500 ppm
Aug 08 02:17:19 dist-test-slave-9gcw krb5kdc[9700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155439, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:19.574932 9715 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:19.576027 9715 webserver.cc:460] Webserver started at http://127.0.0.1:38155/ using document root <none> and password file <none>
I20260808 02:17:19.576609 9715 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:19.576673 9715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:19.576908 9715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:19.579190 9715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "17b8ca13c6bd43d89c2d0249858b753a"
format_stamp: "Formatted at 2026-08-08 02:17:19 on dist-test-slave-9gcw"
server_key: "20d025dc0255b4bd386f81c8e37c87bc"
server_key_iv: "1efcae71dfe97cd9ab1df333d8dfe8bf"
server_key_version: "encryptionkey@0"
I20260808 02:17:19.579830 9715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "17b8ca13c6bd43d89c2d0249858b753a"
format_stamp: "Formatted at 2026-08-08 02:17:19 on dist-test-slave-9gcw"
server_key: "20d025dc0255b4bd386f81c8e37c87bc"
server_key_iv: "1efcae71dfe97cd9ab1df333d8dfe8bf"
server_key_version: "encryptionkey@0"
I20260808 02:17:19.584012 9715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:17:19.587142 9731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:19.588433 9715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:19.588593 9715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "17b8ca13c6bd43d89c2d0249858b753a"
format_stamp: "Formatted at 2026-08-08 02:17:19 on dist-test-slave-9gcw"
server_key: "20d025dc0255b4bd386f81c8e37c87bc"
server_key_iv: "1efcae71dfe97cd9ab1df333d8dfe8bf"
server_key_version: "encryptionkey@0"
I20260808 02:17:19.588702 9715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:19.593556 9715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:19.597021 9715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:19.597272 9715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:19.604856 9715 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35339
I20260808 02:17:19.604882 9787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35339 every 8 connection(s)
I20260808 02:17:19.605885 9715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:19.606647 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 9715
I20260808 02:17:19.606803 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:19.607143 19170 external_mini_cluster.cc:1428] Setting key 0afa0ff6287f9e971245abe2c956ad96
I20260808 02:17:19.609403 9788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:19.612646 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.608641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:59071 (local address 127.0.0.1:35339)
0808 02:17:19.609119 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.609133 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.609233 (+ 100us) server_negotiation.cc:407] Connection header received
0808 02:17:19.609939 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.609957 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.610236 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.610586 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.611449 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.611453 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.611465 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:19.611476 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.611531 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:19.611549 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.611704 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.611774 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.611949 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":165,"threads_started":1}
I20260808 02:17:19.615618 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:19.615679 9788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a: Bootstrap starting.
I20260808 02:17:19.618198 9788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:19.619215 9788 log.cc:824] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:19.619747 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.617848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:54601 (local address 127.0.0.1:35339)
0808 02:17:19.617986 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.617991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.618216 (+ 225us) server_negotiation.cc:407] Connection header received
0808 02:17:19.618412 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.618417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.618470 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.618553 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.619315 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.619321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.619325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:19.619340 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.619373 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:19.619388 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.619479 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.619555 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.619601 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:17:19.621497 9788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a: No bootstrap required, opened a new log
I20260808 02:17:19.625099 9788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b8ca13c6bd43d89c2d0249858b753a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35339 } }
I20260808 02:17:19.625476 9788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:19.625573 9788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17b8ca13c6bd43d89c2d0249858b753a, State: Initialized, Role: FOLLOWER
I20260808 02:17:19.626078 9788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b8ca13c6bd43d89c2d0249858b753a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35339 } }
I20260808 02:17:19.626245 9788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:19.626335 9788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:19.626441 9788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:19.627466 9788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b8ca13c6bd43d89c2d0249858b753a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35339 } }
I20260808 02:17:19.627830 9788 leader_election.cc:302] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17b8ca13c6bd43d89c2d0249858b753a; no voters:
I20260808 02:17:19.628167 9788 leader_election.cc:288] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:19.628311 9798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:19.628628 9798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [term 1 LEADER]: Becoming Leader. State: Replica: 17b8ca13c6bd43d89c2d0249858b753a, State: Running, Role: LEADER
I20260808 02:17:19.629024 9798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b8ca13c6bd43d89c2d0249858b753a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35339 } }
I20260808 02:17:19.629614 9788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:19.630952 9799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17b8ca13c6bd43d89c2d0249858b753a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b8ca13c6bd43d89c2d0249858b753a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35339 } } }
I20260808 02:17:19.630988 9800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17b8ca13c6bd43d89c2d0249858b753a. Latest consensus state: current_term: 1 leader_uuid: "17b8ca13c6bd43d89c2d0249858b753a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b8ca13c6bd43d89c2d0249858b753a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35339 } } }
I20260808 02:17:19.631084 9799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:19.631088 9800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:19.631471 9807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:19.634820 9807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:19.640383 9807 catalog_manager.cc:1347] Generated new cluster ID: b020d9bb3580489cb93e0cc11e6d69de
I20260808 02:17:19.640532 9807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:19.657904 9807 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:19.658284 9807 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:19.666283 9807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17b8ca13c6bd43d89c2d0249858b753a: Generated new TSK 0
I20260808 02:17:19.667088 9807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:19.673497 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 9715
2026-08-08T02:17:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1530 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:19 dist-test-slave-9gcw krb5kdc[9820](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:19 dist-test-slave-9gcw krb5kdc[9820](info): set up 2 sockets
Aug 08 02:17:19 dist-test-slave-9gcw krb5kdc[9820](info): commencing operation
krb5kdc: starting...
W20260808 02:17:20.805549 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:20 dist-test-slave-9gcw krb5kdc[9820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:20.946722 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:21.081522 9833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:21.081825 9833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:21.081869 9833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:21.086115 9833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:21.086182 9833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:21.086203 9833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:21.086220 9833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:21.086267 9833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:21.092756 9833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35967
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:21.094113 9833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:21.095413 9833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:21.105340 9839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:21.105340 9841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:21.105345 9838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:21.105676 9833 server_base.cc:1034] running on GCE node
I20260808 02:17:21.106328 9833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:21.107563 9833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:21.108790 9833 hybrid_clock.cc:648] HybridClock initialized: now 1786155441108769 us; error 46 us; skew 500 ppm
Aug 08 02:17:21 dist-test-slave-9gcw krb5kdc[9820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155441, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:21.111768 9833 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:21.112705 9833 webserver.cc:460] Webserver started at http://127.0.0.1:36969/ using document root <none> and password file <none>
I20260808 02:17:21.113371 9833 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:21.113471 9833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:21.113713 9833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:21.115868 9833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "9d0136727a724d7e90680b11b7a6fd45"
format_stamp: "Formatted at 2026-08-08 02:17:21 on dist-test-slave-9gcw"
server_key: "af79d9e895e77a7d7f0c398050c52560"
server_key_iv: "27087974f176cde27a8d2e76834cfd47"
server_key_version: "encryptionkey@0"
I20260808 02:17:21.116456 9833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "9d0136727a724d7e90680b11b7a6fd45"
format_stamp: "Formatted at 2026-08-08 02:17:21 on dist-test-slave-9gcw"
server_key: "af79d9e895e77a7d7f0c398050c52560"
server_key_iv: "27087974f176cde27a8d2e76834cfd47"
server_key_version: "encryptionkey@0"
I20260808 02:17:21.120508 9833 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:17:21.123656 9848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:21.125036 9833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:21.125238 9833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "9d0136727a724d7e90680b11b7a6fd45"
format_stamp: "Formatted at 2026-08-08 02:17:21 on dist-test-slave-9gcw"
server_key: "af79d9e895e77a7d7f0c398050c52560"
server_key_iv: "27087974f176cde27a8d2e76834cfd47"
server_key_version: "encryptionkey@0"
I20260808 02:17:21.125378 9833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:21.141556 9833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:21.145499 9833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:21.145802 9833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:21.154963 9833 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35967
I20260808 02:17:21.155009 9906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35967 every 8 connection(s)
I20260808 02:17:21.156178 9833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:21.159791 9907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:21.166076 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 9833
I20260808 02:17:21.166227 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:21.166226 9907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45: Bootstrap starting.
I20260808 02:17:21.166556 19170 external_mini_cluster.cc:1428] Setting key 8553f3c2bfcd5057552613aa7aef0f4a
I20260808 02:17:21.169236 9907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:21.170080 9907 log.cc:824] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:21.172219 9907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45: No bootstrap required, opened a new log
Aug 08 02:17:21 dist-test-slave-9gcw krb5kdc[9820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:17:21.176831 9907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0136727a724d7e90680b11b7a6fd45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } }
I20260808 02:17:21.177074 9907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:21.177143 9907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d0136727a724d7e90680b11b7a6fd45, State: Initialized, Role: FOLLOWER
I20260808 02:17:21.177774 9907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0136727a724d7e90680b11b7a6fd45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } }
I20260808 02:17:21.177918 9907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:21.178062 9907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:21.178205 9907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:21.179203 9907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0136727a724d7e90680b11b7a6fd45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } }
I20260808 02:17:21.179587 9907 leader_election.cc:302] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d0136727a724d7e90680b11b7a6fd45; no voters:
I20260808 02:17:21.179980 9907 leader_election.cc:288] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:21.180063 9913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:21.180272 9913 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [term 1 LEADER]: Becoming Leader. State: Replica: 9d0136727a724d7e90680b11b7a6fd45, State: Running, Role: LEADER
I20260808 02:17:21.180667 9913 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0136727a724d7e90680b11b7a6fd45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } }
I20260808 02:17:21.181473 9907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:21.182556 9916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d0136727a724d7e90680b11b7a6fd45. Latest consensus state: current_term: 1 leader_uuid: "9d0136727a724d7e90680b11b7a6fd45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0136727a724d7e90680b11b7a6fd45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } } }
I20260808 02:17:21.182705 9916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:21.182780 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:21.168090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:55747 (local address 127.0.0.1:35967)
0808 02:17:21.168632 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:21.168654 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:21.168700 (+ 46us) server_negotiation.cc:407] Connection header received
0808 02:17:21.169525 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:21.169543 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:21.169895 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:21.170240 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:21.171085 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:21.172093 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:21.173047 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:21.173599 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:21.173774 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:21.176850 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:21.176854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:21.176869 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:21.176897 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:21.178905 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:21.179440 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:21.179446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:21.179450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:21.179563 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:21.179976 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:21.179980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:21.179982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:21.180572 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:21.181378 (+ 806us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:21.181514 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 02:17:21.181770 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":177,"threads_started":1}
I20260808 02:17:21.183066 9915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d0136727a724d7e90680b11b7a6fd45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0136727a724d7e90680b11b7a6fd45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } } }
I20260808 02:17:21.183171 9915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:21.186364 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 02:17:21.187335 9931 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:21.187428 9931 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:21.187529 9924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:21.188536 9924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:21.194849 9924 catalog_manager.cc:1347] Generated new cluster ID: fc923d772fcd4278addc110602b57166
I20260808 02:17:21.195011 9924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:21.198546 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:21.188779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:41795 (local address 127.0.0.1:35967)
0808 02:17:21.188892 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:21.188897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:21.189182 (+ 285us) server_negotiation.cc:407] Connection header received
0808 02:17:21.189317 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:21.189321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:21.189371 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:21.189459 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:21.190366 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:21.191273 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:21.192403 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:21.192771 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:21.192882 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:21.194776 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:21.194780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:21.194783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:21.194812 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:21.196578 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:21.197134 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:21.197160 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:21.197163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:21.197253 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:21.197606 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:21.197609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:21.197611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:21.197834 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:21.197982 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:21.198285 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 02:17:21.198388 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 02:17:21.208446 9924 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:21.209882 9924 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:21.221848 9924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d0136727a724d7e90680b11b7a6fd45: Generated new TSK 0
I20260808 02:17:21.222760 9924 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:21.231782 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 9833
W20260808 02:17:21.240718 9817 connection.cc:535] client connection to 127.0.0.1:35967 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35967): Connection reset by peer (error 104)
2026-08-08T02:17:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1558 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:21 dist-test-slave-9gcw krb5kdc[9941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:21 dist-test-slave-9gcw krb5kdc[9941](info): set up 2 sockets
Aug 08 02:17:21 dist-test-slave-9gcw krb5kdc[9941](info): commencing operation
krb5kdc: starting...
W20260808 02:17:22.460050 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:22 dist-test-slave-9gcw krb5kdc[9941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:22.601688 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34923
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:22.718323 9956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:22.718644 9956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:22.718731 9956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:22.722826 9956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:22.722932 9956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:22.722972 9956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:22.723033 9956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:22.723071 9956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:22.728665 9956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34923
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:22.729905 9956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:22.731003 9956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:22.740046 9961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:22.740046 9964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:22.740056 9962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:22.740339 9956 server_base.cc:1034] running on GCE node
I20260808 02:17:22.741057 9956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:22.742460 9956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:22.743669 9956 hybrid_clock.cc:648] HybridClock initialized: now 1786155442743614 us; error 67 us; skew 500 ppm
Aug 08 02:17:22 dist-test-slave-9gcw krb5kdc[9941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155442, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:22.746800 9956 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:22.747817 9956 webserver.cc:460] Webserver started at http://127.0.0.1:42105/ using document root <none> and password file <none>
I20260808 02:17:22.748446 9956 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:22.748549 9956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:22.748822 9956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:22.751096 9956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "16b493378750475b9e4a67e680ffb4c5"
format_stamp: "Formatted at 2026-08-08 02:17:22 on dist-test-slave-9gcw"
server_key: "8a4f8bdab25e4ba59d986a1e1b51ffa0"
server_key_iv: "819c17e2dbe9575c3f559a8621d7ea10"
server_key_version: "encryptionkey@0"
I20260808 02:17:22.751659 9956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "16b493378750475b9e4a67e680ffb4c5"
format_stamp: "Formatted at 2026-08-08 02:17:22 on dist-test-slave-9gcw"
server_key: "8a4f8bdab25e4ba59d986a1e1b51ffa0"
server_key_iv: "819c17e2dbe9575c3f559a8621d7ea10"
server_key_version: "encryptionkey@0"
I20260808 02:17:22.755784 9956 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:17:22.758426 9972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:22.760061 9956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:17:22.760278 9956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "16b493378750475b9e4a67e680ffb4c5"
format_stamp: "Formatted at 2026-08-08 02:17:22 on dist-test-slave-9gcw"
server_key: "8a4f8bdab25e4ba59d986a1e1b51ffa0"
server_key_iv: "819c17e2dbe9575c3f559a8621d7ea10"
server_key_version: "encryptionkey@0"
I20260808 02:17:22.760429 9956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:22.774101 9956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:22.777406 9956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:22.777626 9956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:22.785760 9956 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34923
I20260808 02:17:22.785792 10029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34923 every 8 connection(s)
I20260808 02:17:22.786815 9956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:22.789850 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 9956
I20260808 02:17:22.789964 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:22.789986 10030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:22.790282 19170 external_mini_cluster.cc:1428] Setting key a065a1f09874618fb7b24034317bd58a
I20260808 02:17:22.798226 10030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5: Bootstrap starting.
I20260808 02:17:22.801266 10030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:22.801879 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:22.791796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:47953 (local address 127.0.0.1:34923)
0808 02:17:22.792344 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:22.792365 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:22.792414 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:17:22.793233 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:22.793259 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:22.793761 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:22.794175 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:22.794634 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:22.796300 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:22.797328 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:22.798092 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:22.798284 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:22.799497 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:22.799505 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:22.799521 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:22.800148 (+ 627us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:22.800306 (+ 158us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:22.800350 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:22.800502 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:22.800708 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 02:17:22.800844 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":131,"threads_started":1}
I20260808 02:17:22.802276 10030 log.cc:824] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:22.804540 10030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5: No bootstrap required, opened a new log
I20260808 02:17:22.804996 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:22.808753 10030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b493378750475b9e4a67e680ffb4c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34923 } }
I20260808 02:17:22.809026 10030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:22.809110 10030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16b493378750475b9e4a67e680ffb4c5, State: Initialized, Role: FOLLOWER
I20260808 02:17:22.809612 10030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b493378750475b9e4a67e680ffb4c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34923 } }
I20260808 02:17:22.809783 10030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:22.809875 10030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:22.809984 10030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:22.811336 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:22.807144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:56043 (local address 127.0.0.1:34923)
0808 02:17:22.807289 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:22.807294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:22.807486 (+ 192us) server_negotiation.cc:407] Connection header received
0808 02:17:22.807644 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:22.807647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:22.807697 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:22.807780 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:22.808067 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:22.808581 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:22.809565 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:22.809846 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:22.809971 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:22.810886 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:22.810888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:22.810891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:22.810902 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:22.810926 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:22.810937 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:22.811008 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:22.811196 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 02:17:22.811228 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 02:17:22.811326 10030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b493378750475b9e4a67e680ffb4c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34923 } }
I20260808 02:17:22.811887 10030 leader_election.cc:302] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16b493378750475b9e4a67e680ffb4c5; no voters:
I20260808 02:17:22.812265 10030 leader_election.cc:288] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:22.812345 10040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:22.812556 10040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [term 1 LEADER]: Becoming Leader. State: Replica: 16b493378750475b9e4a67e680ffb4c5, State: Running, Role: LEADER
I20260808 02:17:22.812880 10040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b493378750475b9e4a67e680ffb4c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34923 } }
I20260808 02:17:22.813741 10030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:22.814199 10043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16b493378750475b9e4a67e680ffb4c5. Latest consensus state: current_term: 1 leader_uuid: "16b493378750475b9e4a67e680ffb4c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b493378750475b9e4a67e680ffb4c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34923 } } }
I20260808 02:17:22.814191 10041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16b493378750475b9e4a67e680ffb4c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b493378750475b9e4a67e680ffb4c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34923 } } }
I20260808 02:17:22.814363 10041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:22.814363 10043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:22.814782 10046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:22.818523 10046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:22.824553 10046 catalog_manager.cc:1347] Generated new cluster ID: cfafd66b8059403597f8f9528d11c3be
I20260808 02:17:22.824708 10046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:22.843029 10046 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:22.844573 10046 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:22.856403 10046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16b493378750475b9e4a67e680ffb4c5: Generated new TSK 0
I20260808 02:17:22.857292 10046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:22.868993 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 9956
W20260808 02:17:22.879099 9939 connection.cc:535] client connection to 127.0.0.1:34923 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34923): Connection reset by peer (error 104)
2026-08-08T02:17:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1637 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:22 dist-test-slave-9gcw krb5kdc[10064](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:22 dist-test-slave-9gcw krb5kdc[10064](info): set up 2 sockets
Aug 08 02:17:22 dist-test-slave-9gcw krb5kdc[10064](info): commencing operation
krb5kdc: starting...
W20260808 02:17:23.972623 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:24 dist-test-slave-9gcw krb5kdc[10064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:24.112895 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44775
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:24.231011 10078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:24.231343 10078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:24.231424 10078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:24.235776 10078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:24.235852 10078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:24.235872 10078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:24.235889 10078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:24.235905 10078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:24.241328 10078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44775
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:24.242408 10078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:24.243462 10078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:24.252545 10086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:24.252547 10083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:24.252542 10084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:24.252638 10078 server_base.cc:1034] running on GCE node
I20260808 02:17:24.253430 10078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:24.254580 10078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:24.255777 10078 hybrid_clock.cc:648] HybridClock initialized: now 1786155444255762 us; error 36 us; skew 500 ppm
Aug 08 02:17:24 dist-test-slave-9gcw krb5kdc[10064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155444, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:24.258751 10078 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:24.259855 10078 webserver.cc:460] Webserver started at http://127.0.0.1:40889/ using document root <none> and password file <none>
I20260808 02:17:24.260419 10078 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:24.260484 10078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:24.260671 10078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:24.263010 10078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "82b0677066714a0b927ac2d83a0dc4dc"
format_stamp: "Formatted at 2026-08-08 02:17:24 on dist-test-slave-9gcw"
server_key: "0ca568027e39383da4a440a7e3c4ec8b"
server_key_iv: "233c007b96fd28e5d9853a4d38e8c96c"
server_key_version: "encryptionkey@0"
I20260808 02:17:24.263574 10078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "82b0677066714a0b927ac2d83a0dc4dc"
format_stamp: "Formatted at 2026-08-08 02:17:24 on dist-test-slave-9gcw"
server_key: "0ca568027e39383da4a440a7e3c4ec8b"
server_key_iv: "233c007b96fd28e5d9853a4d38e8c96c"
server_key_version: "encryptionkey@0"
I20260808 02:17:24.267874 10078 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:17:24.270761 10094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:24.272213 10078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:17:24.272365 10078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "82b0677066714a0b927ac2d83a0dc4dc"
format_stamp: "Formatted at 2026-08-08 02:17:24 on dist-test-slave-9gcw"
server_key: "0ca568027e39383da4a440a7e3c4ec8b"
server_key_iv: "233c007b96fd28e5d9853a4d38e8c96c"
server_key_version: "encryptionkey@0"
I20260808 02:17:24.272500 10078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:24.277590 10078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:24.281037 10078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:24.281288 10078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:24.288903 10078 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44775
I20260808 02:17:24.288964 10150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44775 every 8 connection(s)
I20260808 02:17:24.290091 10078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:24.290369 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 10078
I20260808 02:17:24.290532 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:24.290908 19170 external_mini_cluster.cc:1428] Setting key 268f4228541312178e8e6a8dc9eec6a1
I20260808 02:17:24.293673 10151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:24.296666 10154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:24.292503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:48049 (local address 127.0.0.1:44775)
0808 02:17:24.293037 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:24.293058 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:24.293106 (+ 48us) server_negotiation.cc:407] Connection header received
0808 02:17:24.293846 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:24.293868 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:24.294217 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:24.294577 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:24.295448 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:24.295452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:24.295464 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:24.295475 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:24.295530 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:24.295549 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:24.295704 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:24.295776 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 02:17:24.295941 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":164,"threads_started":1}
I20260808 02:17:24.299571 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:24.300449 10151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc: Bootstrap starting.
I20260808 02:17:24.303356 10151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:24.303558 10154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:24.301834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:35373 (local address 127.0.0.1:44775)
0808 02:17:24.301977 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:24.301982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:24.302120 (+ 138us) server_negotiation.cc:407] Connection header received
0808 02:17:24.302202 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:24.302206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:24.302251 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:24.302345 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:24.303074 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:24.303077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:24.303080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:24.303094 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:24.303125 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:24.303142 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:24.303309 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:24.303383 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 02:17:24.303430 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:17:24.304248 10151 log.cc:824] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:24.306646 10151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc: No bootstrap required, opened a new log
I20260808 02:17:24.310811 10151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b0677066714a0b927ac2d83a0dc4dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44775 } }
I20260808 02:17:24.311084 10151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:24.311136 10151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b0677066714a0b927ac2d83a0dc4dc, State: Initialized, Role: FOLLOWER
I20260808 02:17:24.311560 10151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b0677066714a0b927ac2d83a0dc4dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44775 } }
I20260808 02:17:24.311686 10151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:24.311743 10151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:24.311825 10151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:24.312858 10151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b0677066714a0b927ac2d83a0dc4dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44775 } }
I20260808 02:17:24.313311 10151 leader_election.cc:302] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82b0677066714a0b927ac2d83a0dc4dc; no voters:
I20260808 02:17:24.313633 10151 leader_election.cc:288] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:24.313814 10162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:24.314098 10162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [term 1 LEADER]: Becoming Leader. State: Replica: 82b0677066714a0b927ac2d83a0dc4dc, State: Running, Role: LEADER
I20260808 02:17:24.314474 10162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b0677066714a0b927ac2d83a0dc4dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44775 } }
I20260808 02:17:24.315124 10151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:24.316318 10163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82b0677066714a0b927ac2d83a0dc4dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b0677066714a0b927ac2d83a0dc4dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44775 } } }
I20260808 02:17:24.316457 10163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:24.316874 10173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:24.317389 10164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82b0677066714a0b927ac2d83a0dc4dc. Latest consensus state: current_term: 1 leader_uuid: "82b0677066714a0b927ac2d83a0dc4dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b0677066714a0b927ac2d83a0dc4dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44775 } } }
I20260808 02:17:24.317549 10164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:24.319865 10173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:24.326592 10173 catalog_manager.cc:1347] Generated new cluster ID: 15f6ee292ec042008cad09c73de0cf0e
I20260808 02:17:24.326788 10173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:24.338948 10173 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:24.339341 10173 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:24.346354 10173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82b0677066714a0b927ac2d83a0dc4dc: Generated new TSK 0
I20260808 02:17:24.347247 10173 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:24.353933 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 10078
2026-08-08T02:17:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1482 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:24 dist-test-slave-9gcw krb5kdc[10187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:24 dist-test-slave-9gcw krb5kdc[10187](info): set up 2 sockets
Aug 08 02:17:24 dist-test-slave-9gcw krb5kdc[10187](info): commencing operation
krb5kdc: starting...
W20260808 02:17:25.511767 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:25 dist-test-slave-9gcw krb5kdc[10187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:17:25.657828 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:44669
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:36343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:44669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:25.779393 10201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:25.779735 10201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:25.779819 10201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:25.784125 10201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:25.784230 10201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:25.784271 10201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:25.784307 10201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:25.784348 10201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:25.790072 10201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:36343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:44669
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:44669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:25.791291 10201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:25.792414 10201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:25.801548 10208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:25.801575 10210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:25.801576 10207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:25.801949 10201 server_base.cc:1034] running on GCE node
I20260808 02:17:25.802533 10201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:25.803715 10201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:25.804917 10201 hybrid_clock.cc:648] HybridClock initialized: now 1786155445804892 us; error 38 us; skew 500 ppm
Aug 08 02:17:25 dist-test-slave-9gcw krb5kdc[10187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:25.808059 10201 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:17:25.809077 10201 webserver.cc:460] Webserver started at http://127.18.184.190:35523/ using document root <none> and password file <none>
I20260808 02:17:25.809697 10201 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:25.809790 10201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:25.810039 10201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:25.812124 10201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "c8e67e93a0f645d397b816bdae65dbb8"
format_stamp: "Formatted at 2026-08-08 02:17:25 on dist-test-slave-9gcw"
server_key: "ee667c49e40d2209b6101f381b4d7caa"
server_key_iv: "b9ec94148e338178111d47435520b215"
server_key_version: "encryptionkey@0"
I20260808 02:17:25.812721 10201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "c8e67e93a0f645d397b816bdae65dbb8"
format_stamp: "Formatted at 2026-08-08 02:17:25 on dist-test-slave-9gcw"
server_key: "ee667c49e40d2209b6101f381b4d7caa"
server_key_iv: "b9ec94148e338178111d47435520b215"
server_key_version: "encryptionkey@0"
I20260808 02:17:25.816679 10201 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:17:25.819434 10218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:25.820712 10201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:25.820895 10201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "c8e67e93a0f645d397b816bdae65dbb8"
format_stamp: "Formatted at 2026-08-08 02:17:25 on dist-test-slave-9gcw"
server_key: "ee667c49e40d2209b6101f381b4d7caa"
server_key_iv: "b9ec94148e338178111d47435520b215"
server_key_version: "encryptionkey@0"
I20260808 02:17:25.821024 10201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:25.848436 10201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:25.852124 10201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:25.852344 10201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:25.859781 10201 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:44669
I20260808 02:17:25.859808 10274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:44669 every 8 connection(s)
I20260808 02:17:25.860741 10201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:25.863560 10275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:25.866714 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 10201
I20260808 02:17:25.866918 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:25.867235 19170 external_mini_cluster.cc:1428] Setting key c44c5663ce2708239c3a351231675680
I20260808 02:17:25.870164 10275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8: Bootstrap starting.
I20260808 02:17:25.872576 10275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:25.873448 10275 log.cc:824] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:25.875625 10275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8: No bootstrap required, opened a new log
Aug 08 02:17:25 dist-test-slave-9gcw krb5kdc[10187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:17:25.879940 10275 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8e67e93a0f645d397b816bdae65dbb8" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44669 } }
I20260808 02:17:25.880308 10275 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:25.880375 10275 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8e67e93a0f645d397b816bdae65dbb8, State: Initialized, Role: FOLLOWER
I20260808 02:17:25.880990 10275 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8e67e93a0f645d397b816bdae65dbb8" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44669 } }
I20260808 02:17:25.881219 10275 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:25.881320 10275 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:25.881426 10275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:25.882690 10275 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8e67e93a0f645d397b816bdae65dbb8" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44669 } }
I20260808 02:17:25.883047 10275 leader_election.cc:302] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8e67e93a0f645d397b816bdae65dbb8; no voters:
I20260808 02:17:25.883392 10275 leader_election.cc:288] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:25.883505 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:25.868758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49697 (local address 127.18.184.190:44669)
0808 02:17:25.869313 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:25.869347 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:25.869387 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:17:25.870056 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:25.870074 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:25.870477 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:25.870822 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:25.871727 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:25.872788 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:25.873788 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:25.874534 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:25.874711 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:25.877758 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:25.877763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:25.877777 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:25.877804 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:25.879864 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:25.880559 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:25.880566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:25.880572 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:25.880698 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:25.881125 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:25.881127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:25.881129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:25.881729 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:25.881915 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:25.882249 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 02:17:25.882508 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":160,"threads_started":1}
I20260808 02:17:25.883509 10281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:25.883746 10281 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [term 1 LEADER]: Becoming Leader. State: Replica: c8e67e93a0f645d397b816bdae65dbb8, State: Running, Role: LEADER
I20260808 02:17:25.884052 10281 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8e67e93a0f645d397b816bdae65dbb8" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44669 } }
I20260808 02:17:25.884711 10275 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:25.885372 10284 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8e67e93a0f645d397b816bdae65dbb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8e67e93a0f645d397b816bdae65dbb8" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44669 } } }
I20260808 02:17:25.885529 10284 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:25.885376 10282 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8e67e93a0f645d397b816bdae65dbb8. Latest consensus state: current_term: 1 leader_uuid: "c8e67e93a0f645d397b816bdae65dbb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8e67e93a0f645d397b816bdae65dbb8" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 44669 } } }
I20260808 02:17:25.885665 10282 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:25.886026 10290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:25.887084 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:25.890367 10290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:25.898962 10290 catalog_manager.cc:1347] Generated new cluster ID: efe7dec5a32444ac8e7a24f6405e459d
I20260808 02:17:25.899044 10290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:25.898970 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:25.889425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34683 (local address 127.18.184.190:44669)
0808 02:17:25.889523 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:25.889528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:25.889956 (+ 428us) server_negotiation.cc:407] Connection header received
0808 02:17:25.890162 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:25.890165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:25.890218 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:25.890305 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:25.891367 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:25.891899 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:25.892898 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:25.893302 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:25.893427 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:25.895110 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:25.895115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:25.895118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:25.895148 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:25.896838 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:25.897497 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:25.897500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:25.897502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:25.897594 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:25.897967 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:25.897971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:25.897973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:25.898203 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:25.898370 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:25.898695 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 02:17:25.898806 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:17:25.908502 10290 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:25.909591 10290 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:25.915457 10290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8e67e93a0f645d397b816bdae65dbb8: Generated new TSK 0
I20260808 02:17:25.916198 10290 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:25.921918 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 10201
W20260808 02:17:25.931572 10184 connection.cc:535] client connection to 127.18.184.190:44669 recv error: Network error: failed to read from TLS socket (remote: 127.18.184.190:44669): Connection reset by peer (error 104)
2026-08-08T02:17:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:25 dist-test-slave-9gcw krb5kdc[10310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:25 dist-test-slave-9gcw krb5kdc[10310](info): set up 2 sockets
Aug 08 02:17:25 dist-test-slave-9gcw krb5kdc[10310](info): commencing operation
krb5kdc: starting...
W20260808 02:17:27.040244 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:27 dist-test-slave-9gcw krb5kdc[10310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:17:27.180639 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:36105
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:37213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:36105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:27.298607 10326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:27.298954 10326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:27.299028 10326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:27.303110 10326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:27.303224 10326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:27.303265 10326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:27.303305 10326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:27.303340 10326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:27.308825 10326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:37213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:36105
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:36105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:27.310041 10326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:27.311144 10326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:27.320130 10331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:27.320089 10335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:27.320089 10333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:27.320240 10326 server_base.cc:1034] running on GCE node
I20260808 02:17:27.321035 10326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:27.322266 10326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:27.323460 10326 hybrid_clock.cc:648] HybridClock initialized: now 1786155447323428 us; error 47 us; skew 500 ppm
Aug 08 02:17:27 dist-test-slave-9gcw krb5kdc[10310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155447, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:27.326485 10326 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:17:27.327476 10326 webserver.cc:460] Webserver started at http://127.18.184.190:37093/ using document root <none> and password file <none>
I20260808 02:17:27.328074 10326 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:27.328169 10326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:27.328413 10326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:27.330567 10326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "e442bfb8c49140ecba9652000ec648e1"
format_stamp: "Formatted at 2026-08-08 02:17:27 on dist-test-slave-9gcw"
server_key: "0daac8dafb1ce98213003b4f028912ec"
server_key_iv: "5b580bb6a9088ce67c7d7f9e01b7f494"
server_key_version: "encryptionkey@0"
I20260808 02:17:27.331140 10326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "e442bfb8c49140ecba9652000ec648e1"
format_stamp: "Formatted at 2026-08-08 02:17:27 on dist-test-slave-9gcw"
server_key: "0daac8dafb1ce98213003b4f028912ec"
server_key_iv: "5b580bb6a9088ce67c7d7f9e01b7f494"
server_key_version: "encryptionkey@0"
I20260808 02:17:27.335067 10326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:17:27.337630 10342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:27.339069 10326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:27.339226 10326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "e442bfb8c49140ecba9652000ec648e1"
format_stamp: "Formatted at 2026-08-08 02:17:27 on dist-test-slave-9gcw"
server_key: "0daac8dafb1ce98213003b4f028912ec"
server_key_iv: "5b580bb6a9088ce67c7d7f9e01b7f494"
server_key_version: "encryptionkey@0"
I20260808 02:17:27.339349 10326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:27.362838 10326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:27.366336 10326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:27.366555 10326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:27.373926 10326 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:36105
I20260808 02:17:27.373956 10398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:36105 every 8 connection(s)
I20260808 02:17:27.374912 10326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:27.377791 10399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:27.378820 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 10326
I20260808 02:17:27.378973 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:27.379323 19170 external_mini_cluster.cc:1428] Setting key 2780e2f0d136c3a8392a116528a338c6
I20260808 02:17:27.384161 10399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1: Bootstrap starting.
I20260808 02:17:27.386482 10399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:27.387241 10399 log.cc:824] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:27.388517 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:27.380789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47021 (local address 127.18.184.190:36105)
0808 02:17:27.381311 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:27.381334 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:27.381370 (+ 36us) server_negotiation.cc:407] Connection header received
0808 02:17:27.382027 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:27.382044 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:27.382392 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:27.382722 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:27.383171 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:27.384139 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:27.385058 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:27.385593 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:27.385769 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:27.386701 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:27.386706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:27.386718 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:27.387099 (+ 381us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:27.387153 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:27.387175 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:27.387313 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:27.387678 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 02:17:27.387802 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":142,"threads_started":1}
I20260808 02:17:27.389421 10399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1: No bootstrap required, opened a new log
I20260808 02:17:27.391494 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:27.393566 10399 raft_consensus.cc:348] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e442bfb8c49140ecba9652000ec648e1" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36105 } }
I20260808 02:17:27.393893 10399 raft_consensus.cc:374] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:27.393963 10399 raft_consensus.cc:729] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e442bfb8c49140ecba9652000ec648e1, State: Initialized, Role: FOLLOWER
I20260808 02:17:27.394452 10399 consensus_queue.cc:260] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e442bfb8c49140ecba9652000ec648e1" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36105 } }
I20260808 02:17:27.394616 10399 raft_consensus.cc:388] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:27.394683 10399 raft_consensus.cc:482] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:27.394791 10399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:27.395905 10399 raft_consensus.cc:504] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e442bfb8c49140ecba9652000ec648e1" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36105 } }
I20260808 02:17:27.396248 10399 leader_election.cc:302] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e442bfb8c49140ecba9652000ec648e1; no voters:
I20260808 02:17:27.396598 10399 leader_election.cc:288] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:27.396715 10411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:27.396958 10411 raft_consensus.cc:686] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [term 1 LEADER]: Becoming Leader. State: Replica: e442bfb8c49140ecba9652000ec648e1, State: Running, Role: LEADER
I20260808 02:17:27.397334 10411 consensus_queue.cc:237] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e442bfb8c49140ecba9652000ec648e1" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36105 } }
I20260808 02:17:27.398054 10399 sys_catalog.cc:564] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:27.398459 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:27.393562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34495 (local address 127.18.184.190:36105)
0808 02:17:27.393679 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:27.393683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:27.394094 (+ 411us) server_negotiation.cc:407] Connection header received
0808 02:17:27.394134 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:27.394136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:27.394185 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:27.394259 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:27.394635 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:27.395415 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:27.396337 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:27.396731 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:27.396856 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:27.397913 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:27.397917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:27.397920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:27.397934 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:27.397963 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:27.397975 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:27.398073 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:27.398269 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 02:17:27.398308 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 02:17:27.398679 10411 sys_catalog.cc:455] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e442bfb8c49140ecba9652000ec648e1. Latest consensus state: current_term: 1 leader_uuid: "e442bfb8c49140ecba9652000ec648e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e442bfb8c49140ecba9652000ec648e1" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36105 } } }
I20260808 02:17:27.398801 10411 sys_catalog.cc:458] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:27.400122 10422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:27.400430 10413 sys_catalog.cc:455] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e442bfb8c49140ecba9652000ec648e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e442bfb8c49140ecba9652000ec648e1" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 36105 } } }
I20260808 02:17:27.400554 10413 sys_catalog.cc:458] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:27.403446 10422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:27.408643 10422 catalog_manager.cc:1347] Generated new cluster ID: 8f503c8b44ac4ae790367fd49180918b
I20260808 02:17:27.408713 10422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:27.422488 10422 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:27.423481 10422 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:27.440812 10422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e442bfb8c49140ecba9652000ec648e1: Generated new TSK 0
I20260808 02:17:27.441816 10422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:27.446560 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 10326
W20260808 02:17:27.455118 10308 connection.cc:535] client connection to 127.18.184.190:36105 recv error: Network error: failed to read from TLS socket (remote: 127.18.184.190:36105): Connection reset by peer (error 104)
2026-08-08T02:17:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:27 dist-test-slave-9gcw krb5kdc[10434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:27 dist-test-slave-9gcw krb5kdc[10434](info): set up 2 sockets
Aug 08 02:17:27 dist-test-slave-9gcw krb5kdc[10434](info): commencing operation
krb5kdc: starting...
W20260808 02:17:28.575670 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:28 dist-test-slave-9gcw krb5kdc[10434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:17:28.713411 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:45083
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:45083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:28.831811 10448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:28.832083 10448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:28.832125 10448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:28.836220 10448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:28.836301 10448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:28.836323 10448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:28.836339 10448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:28.836408 10448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:28.842106 10448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:45083
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:45083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:28.843191 10448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:28.844201 10448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:28.853452 10454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:28.853519 10453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:28.853525 10456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:28.853564 10448 server_base.cc:1034] running on GCE node
I20260808 02:17:28.854334 10448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:28.855470 10448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:28.856635 10448 hybrid_clock.cc:648] HybridClock initialized: now 1786155448856608 us; error 42 us; skew 500 ppm
Aug 08 02:17:28 dist-test-slave-9gcw krb5kdc[10434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155448, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:28.859654 10448 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:17:28.860586 10448 webserver.cc:460] Webserver started at http://127.18.184.190:34285/ using document root <none> and password file <none>
I20260808 02:17:28.861110 10448 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:28.861238 10448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:28.861429 10448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:28.863473 10448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "2ea0b76a632849f295b9183afa37ee3d"
format_stamp: "Formatted at 2026-08-08 02:17:28 on dist-test-slave-9gcw"
server_key: "761f4de77d05e2af19a886f1705e536c"
server_key_iv: "c9eb7bc3353289d1c3d9b3cb6706e979"
server_key_version: "encryptionkey@0"
I20260808 02:17:28.863994 10448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "2ea0b76a632849f295b9183afa37ee3d"
format_stamp: "Formatted at 2026-08-08 02:17:28 on dist-test-slave-9gcw"
server_key: "761f4de77d05e2af19a886f1705e536c"
server_key_iv: "c9eb7bc3353289d1c3d9b3cb6706e979"
server_key_version: "encryptionkey@0"
I20260808 02:17:28.868077 10448 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 02:17:28.870805 10464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:28.872259 10448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:17:28.872450 10448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "2ea0b76a632849f295b9183afa37ee3d"
format_stamp: "Formatted at 2026-08-08 02:17:28 on dist-test-slave-9gcw"
server_key: "761f4de77d05e2af19a886f1705e536c"
server_key_iv: "c9eb7bc3353289d1c3d9b3cb6706e979"
server_key_version: "encryptionkey@0"
I20260808 02:17:28.872581 10448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:28.877468 10448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:28.880925 10448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:28.881211 10448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:28.888960 10448 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:45083
I20260808 02:17:28.888970 10521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:45083 every 8 connection(s)
I20260808 02:17:28.890094 10448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:28.891145 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 10448
I20260808 02:17:28.891330 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:28.891665 19170 external_mini_cluster.cc:1428] Setting key 5c3567cd572fc8853382acdb5a747946
I20260808 02:17:28.893745 10522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:28.897119 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:28.893093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48211 (local address 127.18.184.190:45083)
0808 02:17:28.893569 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:28.893584 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:28.893622 (+ 38us) server_negotiation.cc:407] Connection header received
0808 02:17:28.894309 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:28.894327 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:28.894610 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:28.894902 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:28.895811 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:28.895815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:28.895826 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:28.895837 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:28.895892 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:28.895912 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:28.896025 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:28.896137 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 02:17:28.896302 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":140,"threads_started":1}
I20260808 02:17:28.900030 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:28.900115 10522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d: Bootstrap starting.
I20260808 02:17:28.903017 10522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:28.903875 10522 log.cc:824] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:28.904207 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:28.902410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40763 (local address 127.18.184.190:45083)
0808 02:17:28.902540 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:28.902545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:28.902750 (+ 205us) server_negotiation.cc:407] Connection header received
0808 02:17:28.902869 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:28.902875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:28.902920 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:28.903010 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:28.903782 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:28.903785 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:28.903788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:28.903800 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:28.903830 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:28.903842 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:28.903931 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:28.904044 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 02:17:28.904084 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:17:28.906065 10522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d: No bootstrap required, opened a new log
I20260808 02:17:28.909549 10522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0b76a632849f295b9183afa37ee3d" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45083 } }
I20260808 02:17:28.909785 10522 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:28.909828 10522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea0b76a632849f295b9183afa37ee3d, State: Initialized, Role: FOLLOWER
I20260808 02:17:28.910192 10522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0b76a632849f295b9183afa37ee3d" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45083 } }
I20260808 02:17:28.910305 10522 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:28.910346 10522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:28.910416 10522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:28.911445 10522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0b76a632849f295b9183afa37ee3d" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45083 } }
I20260808 02:17:28.911757 10522 leader_election.cc:302] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ea0b76a632849f295b9183afa37ee3d; no voters:
I20260808 02:17:28.912015 10522 leader_election.cc:288] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:28.912159 10532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:28.912428 10532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [term 1 LEADER]: Becoming Leader. State: Replica: 2ea0b76a632849f295b9183afa37ee3d, State: Running, Role: LEADER
I20260808 02:17:28.912755 10532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0b76a632849f295b9183afa37ee3d" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45083 } }
I20260808 02:17:28.913349 10522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:28.914512 10536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ea0b76a632849f295b9183afa37ee3d. Latest consensus state: current_term: 1 leader_uuid: "2ea0b76a632849f295b9183afa37ee3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0b76a632849f295b9183afa37ee3d" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45083 } } }
I20260808 02:17:28.914651 10536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:28.915050 10543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:28.918361 10543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:28.915009 10535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ea0b76a632849f295b9183afa37ee3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0b76a632849f295b9183afa37ee3d" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 45083 } } }
I20260808 02:17:28.918596 10535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:28.924629 10543 catalog_manager.cc:1347] Generated new cluster ID: e143348e097940cca440a4e34dccb66b
I20260808 02:17:28.924834 10543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:28.943024 10543 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:28.943364 10543 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:28.955152 10543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ea0b76a632849f295b9183afa37ee3d: Generated new TSK 0
I20260808 02:17:28.955925 10543 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:28.971119 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 10448
2026-08-08T02:17:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:29 dist-test-slave-9gcw krb5kdc[10558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:29 dist-test-slave-9gcw krb5kdc[10558](info): set up 2 sockets
Aug 08 02:17:29 dist-test-slave-9gcw krb5kdc[10558](info): commencing operation
krb5kdc: starting...
W20260808 02:17:30.077323 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:30 dist-test-slave-9gcw krb5kdc[10558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:17:30.213112 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:42345
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:42345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:30.330647 10571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:30.331008 10571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:30.331095 10571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:30.335526 10571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:30.335647 10571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:30.335685 10571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:30.335726 10571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:30.335773 10571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:30.341509 10571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:42345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:42345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:30.342741 10571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:30.343950 10571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:30.353088 10577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:30.353263 10571 server_base.cc:1034] running on GCE node
W20260808 02:17:30.353075 10579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:30.353076 10581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:30.354003 10571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:30.355159 10571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:30.356370 10571 hybrid_clock.cc:648] HybridClock initialized: now 1786155450356339 us; error 47 us; skew 500 ppm
Aug 08 02:17:30 dist-test-slave-9gcw krb5kdc[10558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:30.359345 10571 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:17:30.360416 10571 webserver.cc:460] Webserver started at http://127.18.184.190:38469/ using document root <none> and password file <none>
I20260808 02:17:30.361007 10571 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:30.361099 10571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:30.361374 10571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:30.363624 10571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "e47f0f21e8eb475ea0e7820d76ef4e40"
format_stamp: "Formatted at 2026-08-08 02:17:30 on dist-test-slave-9gcw"
server_key: "04e1c9776bf3e66451f94172b3ad886d"
server_key_iv: "9ab165e1010095ace779e6e472500926"
server_key_version: "encryptionkey@0"
I20260808 02:17:30.364233 10571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "e47f0f21e8eb475ea0e7820d76ef4e40"
format_stamp: "Formatted at 2026-08-08 02:17:30 on dist-test-slave-9gcw"
server_key: "04e1c9776bf3e66451f94172b3ad886d"
server_key_iv: "9ab165e1010095ace779e6e472500926"
server_key_version: "encryptionkey@0"
I20260808 02:17:30.368114 10571 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 02:17:30.370839 10588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:30.372113 10571 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:17:30.372292 10571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "e47f0f21e8eb475ea0e7820d76ef4e40"
format_stamp: "Formatted at 2026-08-08 02:17:30 on dist-test-slave-9gcw"
server_key: "04e1c9776bf3e66451f94172b3ad886d"
server_key_iv: "9ab165e1010095ace779e6e472500926"
server_key_version: "encryptionkey@0"
I20260808 02:17:30.372426 10571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:30.391993 10571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:30.395255 10571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:30.395498 10571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:30.403184 10571 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:42345
I20260808 02:17:30.403210 10641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:42345 every 8 connection(s)
I20260808 02:17:30.404191 10571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:30.407064 10642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:30.411237 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 10571
I20260808 02:17:30.411389 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:30.411680 19170 external_mini_cluster.cc:1428] Setting key 2ecbe35d41d9cc4e7bd36b589987a247
I20260808 02:17:30.413384 10642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40: Bootstrap starting.
I20260808 02:17:30.416263 10642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:30.417239 10642 log.cc:824] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:30.419349 10642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40: No bootstrap required, opened a new log
Aug 08 02:17:30 dist-test-slave-9gcw krb5kdc[10558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:17:30.423425 10642 raft_consensus.cc:348] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47f0f21e8eb475ea0e7820d76ef4e40" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42345 } }
I20260808 02:17:30.423666 10642 raft_consensus.cc:374] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:30.423738 10642 raft_consensus.cc:729] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47f0f21e8eb475ea0e7820d76ef4e40, State: Initialized, Role: FOLLOWER
I20260808 02:17:30.424220 10642 consensus_queue.cc:260] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47f0f21e8eb475ea0e7820d76ef4e40" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42345 } }
I20260808 02:17:30.424350 10642 raft_consensus.cc:388] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:30.424453 10642 raft_consensus.cc:482] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:30.424589 10642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:30.425997 10642 raft_consensus.cc:504] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47f0f21e8eb475ea0e7820d76ef4e40" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42345 } }
I20260808 02:17:30.426429 10642 leader_election.cc:302] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e47f0f21e8eb475ea0e7820d76ef4e40; no voters:
I20260808 02:17:30.426834 10642 leader_election.cc:288] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:30.426975 10647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:30.427259 10647 raft_consensus.cc:686] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [term 1 LEADER]: Becoming Leader. State: Replica: e47f0f21e8eb475ea0e7820d76ef4e40, State: Running, Role: LEADER
I20260808 02:17:30.427634 10647 consensus_queue.cc:237] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47f0f21e8eb475ea0e7820d76ef4e40" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42345 } }
I20260808 02:17:30.428192 10642 sys_catalog.cc:564] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:30.428311 10645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:30.413296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47923 (local address 127.18.184.190:42345)
0808 02:17:30.413769 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:30.413789 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:30.413843 (+ 54us) server_negotiation.cc:407] Connection header received
0808 02:17:30.414713 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:30.414739 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:30.415185 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:30.415554 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:30.416282 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:30.417785 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:30.418707 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:30.419418 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:30.419581 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:30.422491 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:30.422496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:30.422511 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:30.422539 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:30.424792 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:30.425372 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:30.425378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:30.425383 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:30.425501 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:30.425823 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:30.425825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:30.425826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:30.426310 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:30.426464 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:30.426860 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 02:17:30.427226 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":158,"threads_started":1}
I20260808 02:17:30.428902 10647 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47f0f21e8eb475ea0e7820d76ef4e40. Latest consensus state: current_term: 1 leader_uuid: "e47f0f21e8eb475ea0e7820d76ef4e40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47f0f21e8eb475ea0e7820d76ef4e40" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42345 } } }
I20260808 02:17:30.429569 10648 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e47f0f21e8eb475ea0e7820d76ef4e40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47f0f21e8eb475ea0e7820d76ef4e40" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 42345 } } }
I20260808 02:17:30.429718 10647 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:30.429754 10648 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:30.430212 10657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:30.431419 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:30.433809 10657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:30.441190 10657 catalog_manager.cc:1347] Generated new cluster ID: ca233b62cf424139878a097586556bd2
I20260808 02:17:30.441287 10657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:30.446226 10645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:30.433846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37969 (local address 127.18.184.190:42345)
0808 02:17:30.433984 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:30.433989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:30.437530 (+ 3541us) server_negotiation.cc:407] Connection header received
0808 02:17:30.437697 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:30.437700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:30.437750 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:30.437850 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:30.438695 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:30.439367 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:30.440560 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:30.440895 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:30.440980 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:30.442702 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:30.442714 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:30.442717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:30.442746 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:30.444427 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:30.444911 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:30.444915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:30.444916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:30.444998 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:30.445324 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:30.445327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:30.445328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:30.445530 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:30.445652 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:30.445923 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 02:17:30.446041 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:17:30.449726 10657 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:30.450982 10657 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:30.469642 10657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e47f0f21e8eb475ea0e7820d76ef4e40: Generated new TSK 0
I20260808 02:17:30.470600 10657 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:30.483197 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 10571
2026-08-08T02:17:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:30 dist-test-slave-9gcw krb5kdc[10675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:30 dist-test-slave-9gcw krb5kdc[10675](info): set up 2 sockets
Aug 08 02:17:30 dist-test-slave-9gcw krb5kdc[10675](info): commencing operation
krb5kdc: starting...
W20260808 02:17:31.593385 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:31 dist-test-slave-9gcw krb5kdc[10675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:17:31.732705 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:37965
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:37965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:31.855779 10690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:31.856139 10690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:31.856226 10690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:31.861070 10690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:31.861222 10690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:31.861279 10690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:31.861317 10690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:31.861359 10690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:31.867688 10690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:37965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:37965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:31.868964 10690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:31.870172 10690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:31.879772 10697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:31.879899 10700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:31.879812 10696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:31.880342 10690 server_base.cc:1034] running on GCE node
I20260808 02:17:31.880920 10690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:31.882221 10690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:31.883388 10690 hybrid_clock.cc:648] HybridClock initialized: now 1786155451883369 us; error 34 us; skew 500 ppm
Aug 08 02:17:31 dist-test-slave-9gcw krb5kdc[10675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:31.890992 10690 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:17:31.892059 10690 webserver.cc:460] Webserver started at http://127.18.184.190:45041/ using document root <none> and password file <none>
I20260808 02:17:31.892678 10690 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:31.892791 10690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:31.893054 10690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:31.895311 10690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "268e2ed64aae489595a8c2a2130d5cf0"
format_stamp: "Formatted at 2026-08-08 02:17:31 on dist-test-slave-9gcw"
server_key: "63311531c8f1373503e5d90ea7a51c79"
server_key_iv: "82b235304093b2ee4752160b00f3079f"
server_key_version: "encryptionkey@0"
I20260808 02:17:31.895946 10690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "268e2ed64aae489595a8c2a2130d5cf0"
format_stamp: "Formatted at 2026-08-08 02:17:31 on dist-test-slave-9gcw"
server_key: "63311531c8f1373503e5d90ea7a51c79"
server_key_iv: "82b235304093b2ee4752160b00f3079f"
server_key_version: "encryptionkey@0"
I20260808 02:17:31.900211 10690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 02:17:31.903110 10707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:31.904489 10690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:31.904663 10690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "268e2ed64aae489595a8c2a2130d5cf0"
format_stamp: "Formatted at 2026-08-08 02:17:31 on dist-test-slave-9gcw"
server_key: "63311531c8f1373503e5d90ea7a51c79"
server_key_iv: "82b235304093b2ee4752160b00f3079f"
server_key_version: "encryptionkey@0"
I20260808 02:17:31.904872 10690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:31.925493 10690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:31.929059 10690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:31.929307 10690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:31.937402 10690 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:37965
I20260808 02:17:31.937429 10763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:37965 every 8 connection(s)
I20260808 02:17:31.938490 10690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:31.941102 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 10690
I20260808 02:17:31.941290 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:31.941396 10764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:31.941653 19170 external_mini_cluster.cc:1428] Setting key 491b3f1be2db1d1f29cff3248d8f3653
I20260808 02:17:31.948941 10764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0: Bootstrap starting.
I20260808 02:17:31.951726 10764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:31.952641 10764 log.cc:824] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:31.952798 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:31.943264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39857 (local address 127.18.184.190:37965)
0808 02:17:31.943734 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:31.943754 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:31.943800 (+ 46us) server_negotiation.cc:407] Connection header received
0808 02:17:31.944937 (+ 1137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:31.944963 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:31.945453 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:31.945845 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:31.946267 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:31.947785 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:31.948892 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:31.949588 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:31.949740 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:31.950673 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:31.950680 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:31.950695 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:31.951203 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:31.951274 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:31.951304 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:31.951413 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:31.951581 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 02:17:31.951811 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":128,"threads_started":1}
I20260808 02:17:31.954870 10764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0: No bootstrap required, opened a new log
I20260808 02:17:31.955953 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:31.958937 10764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "268e2ed64aae489595a8c2a2130d5cf0" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37965 } }
I20260808 02:17:31.959203 10764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:31.959303 10764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 268e2ed64aae489595a8c2a2130d5cf0, State: Initialized, Role: FOLLOWER
I20260808 02:17:31.959784 10764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "268e2ed64aae489595a8c2a2130d5cf0" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37965 } }
I20260808 02:17:31.959954 10764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:31.960045 10764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:31.960168 10764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:31.961212 10764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "268e2ed64aae489595a8c2a2130d5cf0" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37965 } }
I20260808 02:17:31.961581 10764 leader_election.cc:302] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 268e2ed64aae489595a8c2a2130d5cf0; no voters:
I20260808 02:17:31.961947 10764 leader_election.cc:288] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:31.962026 10775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:31.962257 10775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [term 1 LEADER]: Becoming Leader. State: Replica: 268e2ed64aae489595a8c2a2130d5cf0, State: Running, Role: LEADER
I20260808 02:17:31.962555 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:31.958133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34957 (local address 127.18.184.190:37965)
0808 02:17:31.958261 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:31.958265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:31.958432 (+ 167us) server_negotiation.cc:407] Connection header received
0808 02:17:31.958552 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:31.958555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:31.958608 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:31.958700 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:31.958978 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:31.959727 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:31.960733 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:31.960999 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:31.961108 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:31.962071 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:31.962074 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:31.962077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:31.962093 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:31.962125 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:31.962140 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:31.962211 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:31.962347 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 02:17:31.962406 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:17:31.962625 10775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "268e2ed64aae489595a8c2a2130d5cf0" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37965 } }
I20260808 02:17:31.963435 10764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:31.964885 10776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "268e2ed64aae489595a8c2a2130d5cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "268e2ed64aae489595a8c2a2130d5cf0" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37965 } } }
I20260808 02:17:31.964929 10777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 268e2ed64aae489595a8c2a2130d5cf0. Latest consensus state: current_term: 1 leader_uuid: "268e2ed64aae489595a8c2a2130d5cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "268e2ed64aae489595a8c2a2130d5cf0" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 37965 } } }
I20260808 02:17:31.965022 10776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:31.965039 10777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:31.965530 10786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:31.968472 10786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:31.974668 10786 catalog_manager.cc:1347] Generated new cluster ID: b78c032c79084397a8fed844829441bf
I20260808 02:17:31.974756 10786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:31.984151 10786 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:31.985268 10786 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:31.989795 10786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 268e2ed64aae489595a8c2a2130d5cf0: Generated new TSK 0
I20260808 02:17:31.990557 10786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:31.999648 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 10690
2026-08-08T02:17:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:32 dist-test-slave-9gcw krb5kdc[10796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:32 dist-test-slave-9gcw krb5kdc[10796](info): set up 2 sockets
Aug 08 02:17:32 dist-test-slave-9gcw krb5kdc[10796](info): commencing operation
krb5kdc: starting...
W20260808 02:17:33.101028 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:33 dist-test-slave-9gcw krb5kdc[10796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:17:33.237030 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:41671
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:44643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:41671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:33.355091 10813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:33.355484 10813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:33.355572 10813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:33.359755 10813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:33.359876 10813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:33.359917 10813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:33.359968 10813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:33.360002 10813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:33.365708 10813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:44643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:41671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:41671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:33.366916 10813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:33.368000 10813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:33.377182 10819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:33.377244 10818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:33.377295 10823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:33.377445 10813 server_base.cc:1034] running on GCE node
I20260808 02:17:33.378144 10813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:33.379377 10813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:33.380549 10813 hybrid_clock.cc:648] HybridClock initialized: now 1786155453380524 us; error 37 us; skew 500 ppm
Aug 08 02:17:33 dist-test-slave-9gcw krb5kdc[10796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155453, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:33.383816 10813 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:17:33.384855 10813 webserver.cc:460] Webserver started at http://127.18.184.190:45321/ using document root <none> and password file <none>
I20260808 02:17:33.385495 10813 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:33.385587 10813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:33.385839 10813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:33.388031 10813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "409d85056cec4d99b660b16855116098"
format_stamp: "Formatted at 2026-08-08 02:17:33 on dist-test-slave-9gcw"
server_key: "8e46cdc10522228d07826351d46314e7"
server_key_iv: "265face89c1b1972bec7b276d7aefab8"
server_key_version: "encryptionkey@0"
I20260808 02:17:33.388612 10813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "409d85056cec4d99b660b16855116098"
format_stamp: "Formatted at 2026-08-08 02:17:33 on dist-test-slave-9gcw"
server_key: "8e46cdc10522228d07826351d46314e7"
server_key_iv: "265face89c1b1972bec7b276d7aefab8"
server_key_version: "encryptionkey@0"
I20260808 02:17:33.392563 10813 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:17:33.395377 10833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:33.396613 10813 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 02:17:33.396780 10813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "409d85056cec4d99b660b16855116098"
format_stamp: "Formatted at 2026-08-08 02:17:33 on dist-test-slave-9gcw"
server_key: "8e46cdc10522228d07826351d46314e7"
server_key_iv: "265face89c1b1972bec7b276d7aefab8"
server_key_version: "encryptionkey@0"
I20260808 02:17:33.396895 10813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:33.401855 10813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:33.405119 10813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:33.405414 10813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:33.412648 10813 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:41671
I20260808 02:17:33.412667 10889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:41671 every 8 connection(s)
I20260808 02:17:33.413702 10813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:33.415122 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 10813
I20260808 02:17:33.415264 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:33.415624 19170 external_mini_cluster.cc:1428] Setting key a46ce7eb2f0808a72da8497bfe493ecd
I20260808 02:17:33.417188 10890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:33.421581 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:33.417110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40979 (local address 127.18.184.190:41671)
0808 02:17:33.417615 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:33.417636 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:33.417679 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:17:33.418507 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:33.418529 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:33.418903 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:33.419267 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:33.420102 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:33.420108 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:33.420124 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:33.420137 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:33.420202 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:33.420228 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:33.420375 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:33.420446 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 02:17:33.420601 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":136,"threads_started":1}
I20260808 02:17:33.424278 10890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098: Bootstrap starting.
I20260808 02:17:33.424546 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:33.426700 10890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:33.427596 10890 log.cc:824] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:33.428486 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:33.426757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35323 (local address 127.18.184.190:41671)
0808 02:17:33.426888 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:33.426893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:33.427082 (+ 189us) server_negotiation.cc:407] Connection header received
0808 02:17:33.427175 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:33.427178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:33.427217 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:33.427295 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:33.428031 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:33.428034 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:33.428037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:33.428048 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:33.428098 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:33.428110 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:33.428208 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:33.428322 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 02:17:33.428364 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:17:33.429908 10890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098: No bootstrap required, opened a new log
I20260808 02:17:33.433630 10890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409d85056cec4d99b660b16855116098" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41671 } }
I20260808 02:17:33.433871 10890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:33.433945 10890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409d85056cec4d99b660b16855116098, State: Initialized, Role: FOLLOWER
I20260808 02:17:33.434387 10890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409d85056cec4d99b660b16855116098" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41671 } }
I20260808 02:17:33.434531 10890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:33.434612 10890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:33.434733 10890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:33.435719 10890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409d85056cec4d99b660b16855116098" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41671 } }
I20260808 02:17:33.436087 10890 leader_election.cc:302] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 409d85056cec4d99b660b16855116098; no voters:
I20260808 02:17:33.436470 10890 leader_election.cc:288] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:33.436658 10904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:33.437021 10904 raft_consensus.cc:686] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [term 1 LEADER]: Becoming Leader. State: Replica: 409d85056cec4d99b660b16855116098, State: Running, Role: LEADER
I20260808 02:17:33.437393 10904 consensus_queue.cc:237] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409d85056cec4d99b660b16855116098" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41671 } }
I20260808 02:17:33.437804 10890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:33.439044 10906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 409d85056cec4d99b660b16855116098. Latest consensus state: current_term: 1 leader_uuid: "409d85056cec4d99b660b16855116098" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409d85056cec4d99b660b16855116098" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41671 } } }
I20260808 02:17:33.439180 10906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:33.439499 10905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "409d85056cec4d99b660b16855116098" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409d85056cec4d99b660b16855116098" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 41671 } } }
I20260808 02:17:33.439586 10905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:33.439599 10913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:33.442905 10913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:33.448712 10913 catalog_manager.cc:1347] Generated new cluster ID: 6a8623a1b1f74857838e6dacda312cce
I20260808 02:17:33.448791 10913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:33.463266 10913 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:33.463624 10913 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:33.475090 10913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 409d85056cec4d99b660b16855116098: Generated new TSK 0
I20260808 02:17:33.475940 10913 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:33.480631 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 10813
2026-08-08T02:17:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1481 ms)
[----------] 18 tests from AuthTokenIssuingTest (27561 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:33 dist-test-slave-9gcw krb5kdc[10925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:33 dist-test-slave-9gcw krb5kdc[10925](info): set up 2 sockets
Aug 08 02:17:33 dist-test-slave-9gcw krb5kdc[10925](info): commencing operation
krb5kdc: starting...
W20260808 02:17:34.593431 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:34 dist-test-slave-9gcw krb5kdc[10925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:17:34.740881 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:43825
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:43825,127.18.184.189:43589,127.18.184.188:35747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:34.866463 10940 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:17:34.866887 10940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:34.866963 10940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:34.867061 10940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:34.871768 10940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:34.871882 10940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:34.871924 10940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:34.871968 10940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:34.872005 10940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:34.878516 10940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:43825,127.18.184.189:43589,127.18.184.188:35747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:43825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:34.879858 10940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:34.881011 10940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:34.890254 10948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:34.890282 10947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:34.890445 10940 server_base.cc:1034] running on GCE node
W20260808 02:17:34.890617 10950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:34.891268 10940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:34.892524 10940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:34.893759 10940 hybrid_clock.cc:648] HybridClock initialized: now 1786155454893740 us; error 44 us; skew 500 ppm
Aug 08 02:17:34 dist-test-slave-9gcw krb5kdc[10925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155454, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:34.896924 10940 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:17:34.898010 10940 webserver.cc:460] Webserver started at http://127.18.184.190:40697/ using document root <none> and password file <none>
I20260808 02:17:34.898664 10940 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:34.898762 10940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:34.899029 10940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:34.901448 10940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "cecfd75219624e3483fdeff41cbcd954"
format_stamp: "Formatted at 2026-08-08 02:17:34 on dist-test-slave-9gcw"
server_key: "4001a4ebcc0e6c8b4428180b61dd787f"
server_key_iv: "76587029043ddde13b517919640dc9c9"
server_key_version: "encryptionkey@0"
I20260808 02:17:34.902118 10940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "cecfd75219624e3483fdeff41cbcd954"
format_stamp: "Formatted at 2026-08-08 02:17:34 on dist-test-slave-9gcw"
server_key: "4001a4ebcc0e6c8b4428180b61dd787f"
server_key_iv: "76587029043ddde13b517919640dc9c9"
server_key_version: "encryptionkey@0"
I20260808 02:17:34.906343 10940 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:17:34.909221 10958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:34.910848 10940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:34.911031 10940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "cecfd75219624e3483fdeff41cbcd954"
format_stamp: "Formatted at 2026-08-08 02:17:34 on dist-test-slave-9gcw"
server_key: "4001a4ebcc0e6c8b4428180b61dd787f"
server_key_iv: "76587029043ddde13b517919640dc9c9"
server_key_version: "encryptionkey@0"
I20260808 02:17:34.911170 10940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:34.926352 10940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:34.931074 10940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:34.931397 10940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:34.940529 10940 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:43825
I20260808 02:17:34.940881 11014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:43825 every 8 connection(s)
I20260808 02:17:34.941756 10940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:34.944813 11015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:34.949484 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 10940
I20260808 02:17:34.949285 11015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:34.949622 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:34.949972 19170 external_mini_cluster.cc:1428] Setting key 6a2b8ec1e62446a16e0232214bf75255
WARNING: no policy specified for kudu/127.18.184.189@KRBTEST.COM; defaulting to no policy
Aug 08 02:17:34 dist-test-slave-9gcw krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155454, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.189@KRBTEST.COM" created.
I20260808 02:17:34.967423 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:34.951943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.18.184.190:43825)
0808 02:17:34.952647 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:34.952664 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:34.952790 (+ 126us) server_negotiation.cc:407] Connection header received
0808 02:17:34.953973 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:34.953978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:34.954209 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:34.954384 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:34.955667 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:34.956666 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:34.958333 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:34.958743 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:34.958883 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:34.961876 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:34.961882 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:34.961892 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:34.961951 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:34.963815 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:34.964425 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:34.964432 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:34.964435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:34.964605 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:34.964961 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:34.964967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:34.964972 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:34.965605 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:34.965769 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:34.966254 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 02:17:34.966576 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":558,"spinlock_wait_cycles":16640,"thread_start_us":229,"threads_started":1}
I20260808 02:17:34.967427 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:34.951944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43825 (local address 127.0.0.1:36844)
0808 02:17:34.952593 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:34.952648 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:34.953674 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:34.954445 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:34.954454 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:34.955012 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:34.955467 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:34.955497 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:34.956821 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:34.956828 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:34.958190 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:34.958205 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:34.958929 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:34.958934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:34.959072 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:34.959727 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:34.959756 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:34.961692 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:34.963966 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:34.963978 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:34.963991 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:34.964267 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:34.964727 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:34.964729 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:34.964730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:34.964855 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:34.965780 (+ 925us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:34.965786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:34.965949 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 02:17:34.966099 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 02:17:34.966686 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":149,"thread_start_us":166,"threads_started":1}
I20260808 02:17:34.969962 11015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:34.970752 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:34.970475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:43589 (local address 127.0.0.1:38862)
0808 02:17:34.970601 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:34.970688 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 02:17:34.970943 10961 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
I20260808 02:17:34.971858 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:34.971634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:43589 (local address 127.0.0.1:38870)
0808 02:17:34.971745 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:34.971801 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 02:17:34.972003 11015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.189:43589: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.189.keytab.
Entry for principal kudu/127.18.184.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.189.keytab.
I20260808 02:17:34.984015 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.189:43589
--webserver_interface=127.18.184.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.189.keytab
--principal=kudu/127.18.184.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:43825,127.18.184.189:43589,127.18.184.188:35747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:17:35.021258 11015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } attempt: 1
I20260808 02:17:35.022213 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.021895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:43589 (local address 127.0.0.1:38884)
0808 02:17:35.022056 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.022132 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:17:35.023063 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.022853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:43589 (local address 127.0.0.1:38898)
0808 02:17:35.022960 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.023008 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 02:17:35.023255 11015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.189:43589: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
I20260808 02:17:35.091452 11015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } attempt: 2
I20260808 02:17:35.092494 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.092133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:43589 (local address 127.0.0.1:38900)
0808 02:17:35.092338 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.092409 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260808 02:17:35.093518 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.093294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:43589 (local address 127.0.0.1:38908)
0808 02:17:35.093404 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.093455 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 02:17:35.093664 11015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.189:43589: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
W20260808 02:17:35.110294 11022 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:17:35.110605 11022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:35.110662 11022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:35.110755 11022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:35.115228 11022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:35.115300 11022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:35.115321 11022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:35.115339 11022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:35.115381 11022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:35.121107 11022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:43825,127.18.184.189:43589,127.18.184.188:35747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.189.keytab
--principal=kudu/127.18.184.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.189:43589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.184.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:35.122409 11022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:35.123513 11022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:35.132718 11030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:35.132838 11028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:35.132718 11027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:35.133107 11022 server_base.cc:1034] running on GCE node
I20260808 02:17:35.133777 11022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:35.135004 11022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:35.136235 11022 hybrid_clock.cc:648] HybridClock initialized: now 1786155455136211 us; error 39 us; skew 500 ppm
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:35.139420 11022 init.cc:377] Logged in from keytab as kudu/127.18.184.189@KRBTEST.COM (short username kudu)
I20260808 02:17:35.140414 11022 webserver.cc:460] Webserver started at http://127.18.184.189:39757/ using document root <none> and password file <none>
I20260808 02:17:35.141016 11022 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:35.141111 11022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:35.141407 11022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:35.143658 11022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/data/instance:
uuid: "112998d8872c4605beac8084bcd68834"
format_stamp: "Formatted at 2026-08-08 02:17:35 on dist-test-slave-9gcw"
server_key: "05acd7370f7b03ed1c983936d82ca715"
server_key_iv: "af476f02d1e70f670fb7c3b570c9afd7"
server_key_version: "encryptionkey@0"
I20260808 02:17:35.144270 11022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/wal/instance:
uuid: "112998d8872c4605beac8084bcd68834"
format_stamp: "Formatted at 2026-08-08 02:17:35 on dist-test-slave-9gcw"
server_key: "05acd7370f7b03ed1c983936d82ca715"
server_key_iv: "af476f02d1e70f670fb7c3b570c9afd7"
server_key_version: "encryptionkey@0"
I20260808 02:17:35.148377 11022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 02:17:35.151176 11039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:35.152524 11022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:17:35.152675 11022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/wal
uuid: "112998d8872c4605beac8084bcd68834"
format_stamp: "Formatted at 2026-08-08 02:17:35 on dist-test-slave-9gcw"
server_key: "05acd7370f7b03ed1c983936d82ca715"
server_key_iv: "af476f02d1e70f670fb7c3b570c9afd7"
server_key_version: "encryptionkey@0"
I20260808 02:17:35.152801 11022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:35.178115 11022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:35.181473 11022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:35.181705 11022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:35.184875 11015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } attempt: 3
I20260808 02:17:35.185761 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.185413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:43589 (local address 127.0.0.1:38914)
0808 02:17:35.185557 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.185654 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 02:17:35.186641 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.186341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:43589 (local address 127.0.0.1:38926)
0808 02:17:35.186487 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.186556 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 02:17:35.186800 11015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.189:43589: Network error: Client connection negotiation failed: client connection to 127.18.184.189:43589: connect: Connection refused (error 111)
I20260808 02:17:35.189504 11022 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.189:43589
I20260808 02:17:35.189831 11099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.189:43589 every 8 connection(s)
I20260808 02:17:35.190518 11022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
I20260808 02:17:35.192632 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 11022
I20260808 02:17:35.192798 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-1/wal/instance
I20260808 02:17:35.193107 19170 external_mini_cluster.cc:1428] Setting key 2f86fd1d255129c736b2131cf2068d3f
I20260808 02:17:35.193603 11100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:35.197695 11100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.184.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.188@KRBTEST.COM" created.
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.189@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:17:35.214418 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.199887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36852 (local address 127.18.184.190:43825)
0808 02:17:35.200039 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.200043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.201259 (+ 1216us) server_negotiation.cc:407] Connection header received
0808 02:17:35.202411 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.202415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.202487 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.202588 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.203997 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.204552 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.206029 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.206284 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.206402 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.209166 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.209179 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.209185 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.209225 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.211840 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.212445 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.212453 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.212456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.212565 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.212924 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.212930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.212933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.213265 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.213384 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.214039 (+ 655us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.214207 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:17:35.215376 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.200419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43825 (local address 127.0.0.1:36852)
0808 02:17:35.201034 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.201092 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:35.202170 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:35.202777 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:35.202784 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:35.203201 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:35.203814 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.203833 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.204698 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.204701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.205902 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.205912 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.206452 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.206456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.206640 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.207283 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:35.207301 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:35.208949 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:35.212001 (+ 3052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.212007 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.212021 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.212336 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.212678 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.212681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.212682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.212830 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.213425 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:35.213431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:35.213728 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 02:17:35.213945 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 02:17:35.214260 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":188,"threads_started":1}
I20260808 02:17:35.216979 11100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.189@KRBTEST.COM for kudu/127.18.184.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.188.keytab.
Entry for principal kudu/127.18.184.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.188.keytab.
I20260808 02:17:35.225481 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.188:35747
--webserver_interface=127.18.184.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.188.keytab
--principal=kudu/127.18.184.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:33311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:43825,127.18.184.189:43589,127.18.184.188:35747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:17:35.227571 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.217600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:43589 (local address 127.0.0.1:38930)
0808 02:17:35.217791 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.217807 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:35.217900 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:35.218595 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:35.218601 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:35.218766 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:35.218863 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.218869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.220016 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.220019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.220605 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.220614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.221363 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.221366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.221447 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.221963 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:35.221975 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:35.223312 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:35.225589 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.225593 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.225595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.225865 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.226159 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.226164 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.226169 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.226295 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.226847 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:35.226852 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:35.226977 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 02:17:35.227104 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 02:17:35.227300 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260808 02:17:35.227758 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.217812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38930 (local address 127.18.184.189:43589)
0808 02:17:35.218140 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.218149 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.218169 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:17:35.218272 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.218277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.218431 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.218545 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.218973 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.219864 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.220783 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.221236 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.221328 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.223445 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.223455 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.223466 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.223488 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.225470 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.225979 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.225984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.225985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.226066 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.226392 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.226395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.226396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.226661 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.226856 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.227256 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.227430 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":119,"threads_started":1}
I20260808 02:17:35.230242 11100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:35.231197 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.230873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:56980)
0808 02:17:35.231031 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.231127 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260808 02:17:35.231473 11043 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
I20260808 02:17:35.232391 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.232158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:56994)
0808 02:17:35.232284 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.232334 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 02:17:35.232555 11100 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:35747: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
I20260808 02:17:35.281808 11100 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } attempt: 1
I20260808 02:17:35.282788 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.282465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:57006)
0808 02:17:35.282632 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.282700 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 02:17:35.283711 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.283415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:57010)
0808 02:17:35.283556 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.283629 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 02:17:35.283923 11100 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:35747: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
I20260808 02:17:35.329985 11015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } attempt: 4
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155454, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.189@KRBTEST.COM
I20260808 02:17:35.339895 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.330734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38940 (local address 127.18.184.189:43589)
0808 02:17:35.330897 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.330903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.330998 (+ 95us) server_negotiation.cc:407] Connection header received
0808 02:17:35.331179 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.331182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.331229 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.331322 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.331840 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.332822 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.333722 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.333976 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.334094 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.336321 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.336324 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.336326 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.336353 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.337958 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.338495 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.338499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.338500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.338585 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.338926 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.338931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.338933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.339139 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.339260 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.339578 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.339721 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:17:35.339895 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.330663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:43589 (local address 127.0.0.1:38940)
0808 02:17:35.330896 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.330916 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:35.331060 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:35.331358 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:35.331374 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:35.331576 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:35.331697 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.331704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.332955 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.332961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.333583 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.333592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.334105 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.334110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.334229 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.334799 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:35.334811 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:35.336165 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:35.338096 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.338101 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.338103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.338366 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.338700 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.338703 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.338706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.338823 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.339264 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:35.339269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:35.339379 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 02:17:35.339517 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 02:17:35.339694 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260808 02:17:35.341179 11015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:35.342023 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.341723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:57022)
0808 02:17:35.341899 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.341953 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 02:17:35.342876 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.342567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:57036)
0808 02:17:35.342702 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.342801 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 02:17:35.343056 11015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:35747: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
W20260808 02:17:35.351076 11106 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:17:35.351468 11106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:35.351532 11106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:35.351609 11106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 02:17:35.352151 11100 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } attempt: 2
I20260808 02:17:35.353111 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.352753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:57038)
0808 02:17:35.352933 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.353012 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 02:17:35.354013 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.353778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:57044)
0808 02:17:35.353890 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.353944 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 02:17:35.354169 11100 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:35747: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
W20260808 02:17:35.356536 11106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:35.356594 11106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:35.356614 11106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:35.356632 11106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:35.356655 11106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:35.362447 11106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:33311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:43825,127.18.184.189:43589,127.18.184.188:35747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.188.keytab
--principal=kudu/127.18.184.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.188:35747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.184.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:35.363683 11106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:35.364898 11106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:35.374279 11115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:35.374270 11113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:35.374262 11112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:35.374372 11106 server_base.cc:1034] running on GCE node
I20260808 02:17:35.375528 11106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:35.376947 11106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:35.378188 11106 hybrid_clock.cc:648] HybridClock initialized: now 1786155455378165 us; error 41 us; skew 500 ppm
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:35.381714 11106 init.cc:377] Logged in from keytab as kudu/127.18.184.188@KRBTEST.COM (short username kudu)
I20260808 02:17:35.382747 11106 webserver.cc:460] Webserver started at http://127.18.184.188:36535/ using document root <none> and password file <none>
I20260808 02:17:35.383342 11106 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:35.383433 11106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:35.383674 11106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:35.385872 11106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/data/instance:
uuid: "bff8bc1acf8f45c2a6557d67b3459c4a"
format_stamp: "Formatted at 2026-08-08 02:17:35 on dist-test-slave-9gcw"
server_key: "8e721d37f841d5222b118788e964374c"
server_key_iv: "f81e8e206ec64e55aa2e1b2c27b91d69"
server_key_version: "encryptionkey@0"
I20260808 02:17:35.386514 11106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/wal/instance:
uuid: "bff8bc1acf8f45c2a6557d67b3459c4a"
format_stamp: "Formatted at 2026-08-08 02:17:35 on dist-test-slave-9gcw"
server_key: "8e721d37f841d5222b118788e964374c"
server_key_iv: "f81e8e206ec64e55aa2e1b2c27b91d69"
server_key_version: "encryptionkey@0"
I20260808 02:17:35.390609 11106 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:17:35.393374 11122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:35.394819 11106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:35.394986 11106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/wal
uuid: "bff8bc1acf8f45c2a6557d67b3459c4a"
format_stamp: "Formatted at 2026-08-08 02:17:35 on dist-test-slave-9gcw"
server_key: "8e721d37f841d5222b118788e964374c"
server_key_iv: "f81e8e206ec64e55aa2e1b2c27b91d69"
server_key_version: "encryptionkey@0"
I20260808 02:17:35.395116 11106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:35.402212 11015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } attempt: 1
I20260808 02:17:35.403085 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.402753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:57050)
0808 02:17:35.402927 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.402995 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 02:17:35.404021 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.403767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:57062)
0808 02:17:35.403906 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.403957 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 02:17:35.404163 11015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:35747: Network error: Client connection negotiation failed: client connection to 127.18.184.188:35747: connect: Connection refused (error 111)
I20260808 02:17:35.422788 11106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:35.426512 11106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:35.426749 11106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:35.434314 11106 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.188:35747
I20260808 02:17:35.434623 11181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.188:35747 every 8 connection(s)
I20260808 02:17:35.435401 11106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
I20260808 02:17:35.438361 11182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:35.442337 11182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:35.445351 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 11106
I20260808 02:17:35.445328 11100 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } attempt: 3
I20260808 02:17:35.445494 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155161841348-19170-0/minicluster-data/master-2/wal/instance
I20260808 02:17:35.445808 19170 external_mini_cluster.cc:1428] Setting key a458371dd26bff08013bada2c34e1d66
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.189@KRBTEST.COM for kudu/127.18.184.188@KRBTEST.COM
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.188@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.188@KRBTEST.COM
I20260808 02:17:35.457510 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.445777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:57066)
0808 02:17:35.445961 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.445975 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:35.446051 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:35.447144 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:35.447147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:35.447369 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:35.447480 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.447488 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.449130 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.449133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.449817 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.449829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.450491 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.450495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.450578 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.451159 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:35.451176 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:35.452678 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:35.455162 (+ 2484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.455165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.455167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.455468 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.455916 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.455920 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.455922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.456037 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.456884 (+ 847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:35.456887 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:35.457007 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 02:17:35.457116 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 02:17:35.457291 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260808 02:17:35.458169 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.444313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36866 (local address 127.18.184.190:43825)
0808 02:17:35.444478 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.444483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.445566 (+ 1083us) server_negotiation.cc:407] Connection header received
0808 02:17:35.446948 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.446954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.447017 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.447116 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.448001 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.448761 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.450032 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.450407 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.450516 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.453743 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.453747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.453749 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.453997 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.455964 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.456515 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.456518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.456520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.456608 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.456938 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.456941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.456943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.457230 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.457326 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.457862 (+ 536us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.457991 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:17:35.458806 11184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.444702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:43825 (local address 127.0.0.1:36866)
0808 02:17:35.445386 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.445448 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:35.446711 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:35.447175 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:35.447180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:35.447532 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:35.447784 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.447795 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.448891 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.448894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.449913 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.449926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.450695 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.450699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.450844 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.451784 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:35.451820 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:35.453593 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:35.456072 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.456083 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.456098 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.456386 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.456730 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.456733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.456735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.456827 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.457429 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:35.457436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:35.457558 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 02:17:35.458318 (+ 760us) client_negotiation.cc:240] Negotiation successful
0808 02:17:35.458519 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":131,"threads_started":1}
I20260808 02:17:35.458817 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.445932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57066 (local address 127.18.184.188:35747)
0808 02:17:35.446332 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.446343 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.446383 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:17:35.446711 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.446733 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.446972 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.447158 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.447640 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.449019 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.449945 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.450365 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.450541 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.452836 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.452842 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.452853 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.452874 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.455029 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.455619 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.455625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.455632 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.455788 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.456160 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.456163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.456166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.456766 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.456906 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.457314 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.457651 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":292,"thread_start_us":156,"threads_started":1}
I20260808 02:17:35.459132 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.446458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57076 (local address 127.18.184.188:35747)
0808 02:17:35.447662 (+ 1204us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.447674 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.447689 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:17:35.447718 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.447720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.447759 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.447864 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.448645 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.449438 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.450629 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.451000 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.451118 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.454469 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.454480 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.454482 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.454504 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.456906 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.457407 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.457411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.457413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.457496 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.457829 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.457835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.457838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.458029 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.458193 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.458852 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.458965 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1135,"thread_start_us":111,"threads_started":1}
I20260808 02:17:35.460134 11182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:35.463532 11100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834: Bootstrap starting.
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.188@KRBTEST.COM for kudu/127.18.184.189@KRBTEST.COM
I20260808 02:17:35.466328 11100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834: Neither blocks nor log segments found. Creating new log.
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:17:35.467346 11100 log.cc:824] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:35.469878 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.460910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.18.184.189:43589)
0808 02:17:35.461020 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.461024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.461038 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:17:35.461121 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.461124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.461215 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.461297 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.461857 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.462568 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.463440 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.463667 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.463773 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.466036 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.466041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.466044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.466077 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.467893 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.468481 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.468484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.468486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.468570 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.468861 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.468865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.468866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.469208 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.469315 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.469614 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.469730 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:17:35.469877 11184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.460670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:43589 (local address 127.0.0.1:38956)
0808 02:17:35.460936 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.460952 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:35.461031 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:35.461420 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:35.461424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:35.461596 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:35.461722 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.461730 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.462730 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.462734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.463322 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.463331 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.463778 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.463781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.463880 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.464365 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:35.464383 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:35.465846 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:35.468010 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.468014 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.468016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.468357 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.468668 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.468670 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.468671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.468752 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.469331 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:35.469335 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:35.469426 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 02:17:35.469535 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 02:17:35.469690 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260808 02:17:35.469901 11100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834: No bootstrap required, opened a new log
I20260808 02:17:35.470885 11182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:35.471306 11015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } attempt: 2
I20260808 02:17:35.472056 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.461314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36882 (local address 127.18.184.190:43825)
0808 02:17:35.461424 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.461428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.461443 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:17:35.461482 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.461484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.461524 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.461607 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.462280 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.462845 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.463890 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.464185 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.464303 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.468009 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.468013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.468016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.468041 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.469962 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.470477 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.470481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.470483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.470580 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.470924 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.470927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.470929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.471120 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.471353 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.471728 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.471885 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:17:35.475567 11100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } }
I20260808 02:17:35.475943 11100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 112998d8872c4605beac8084bcd68834, State: Initialized, Role: FOLLOWER
I20260808 02:17:35.476352 11100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } }
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.188@KRBTEST.COM for kudu/127.18.184.188@KRBTEST.COM
I20260808 02:17:35.476984 11190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } } }
I20260808 02:17:35.477133 11190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155454, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.188@KRBTEST.COM
I20260808 02:17:35.478003 11100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 02:17:35 dist-test-slave-9gcw krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.189@KRBTEST.COM
I20260808 02:17:35.482138 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.471669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:57082)
0808 02:17:35.471835 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.471849 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:35.471917 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:35.472449 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:35.472452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:35.472626 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:35.472718 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.472724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.473699 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.473703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.474497 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.474506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.475399 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.475405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.475508 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.475929 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:35.475947 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:35.477860 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:35.480194 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.480198 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.480200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.480482 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.480837 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.480841 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.480843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.480941 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.481514 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:35.481518 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:35.481632 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 02:17:35.481765 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 02:17:35.481924 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 02:17:35.482331 11184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.471395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:35747 (local address 127.0.0.1:57078)
0808 02:17:35.471779 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:35.471792 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:35.471869 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:35.472157 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:35.472159 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:35.472347 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:35.472440 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.472449 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.473457 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.473460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.474249 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:35.474259 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.474882 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.474887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:35.474969 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.475494 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:35.475508 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:35.477862 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:35.480585 (+ 2723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.480588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.480591 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.480857 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.481194 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:35.481199 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:35.481202 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:35.481286 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:35.481752 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:35.481755 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:35.481852 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 02:17:35.481968 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 02:17:35.482162 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342}
I20260808 02:17:35.482745 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.471605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57078 (local address 127.18.184.188:35747)
0808 02:17:35.471721 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.471725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.471940 (+ 215us) server_negotiation.cc:407] Connection header received
0808 02:17:35.471966 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.471968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.472001 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.472055 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.472568 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.473351 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.474376 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.474701 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.474782 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.478000 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.478004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.478006 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.478029 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.480465 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.480961 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.480964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.480966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.481052 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.481416 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.481420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.481423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.481640 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.481750 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.482544 (+ 794us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.482654 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 02:17:35.483424 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.472110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57082 (local address 127.18.184.188:35747)
0808 02:17:35.472225 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.472229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.472240 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:17:35.472271 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.472274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.472313 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.472433 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.472844 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.473582 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.474622 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.475236 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.475728 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.478071 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.478076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.478078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.478097 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.480059 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.480608 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.480615 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.480619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.480718 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.481052 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.481055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.481057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.481371 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.481463 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.483130 (+ 1667us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.483253 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":49}
I20260808 02:17:35.485219 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.474033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38972 (local address 127.18.184.189:43589)
0808 02:17:35.474133 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.474137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.474301 (+ 164us) server_negotiation.cc:407] Connection header received
0808 02:17:35.474466 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.474470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.474518 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.474566 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.475441 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.476112 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.477180 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.477488 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.477589 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.481057 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.481060 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.481063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.481089 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.482905 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.483415 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.483419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.483421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.483512 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.483862 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.483865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.483867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.484078 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.484248 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.484923 (+ 675us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.485052 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 02:17:35.486454 11182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a: Bootstrap starting.
I20260808 02:17:35.486862 11015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954: Bootstrap starting.
I20260808 02:17:35.489073 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:35.489276 11182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:35.489640 11015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:35.490217 11182 log.cc:824] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:35.490550 11015 log.cc:824] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:35.492156 11182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a: No bootstrap required, opened a new log
I20260808 02:17:35.489452 11022 master_runner.cc:186] Error getting master registration for 127.18.184.190:43825: OK, instance_id { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" instance_seqno: 1786155454938016 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:17:35.493484 10987 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cecfd75219624e3483fdeff41cbcd954"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36882
W20260808 02:17:35.494562 11201 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:35.495026 11201 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:35.495067 11022 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:17:35.496348 11182 raft_consensus.cc:348] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } }
I20260808 02:17:35.496634 11182 raft_consensus.cc:729] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff8bc1acf8f45c2a6557d67b3459c4a, State: Initialized, Role: FOLLOWER
I20260808 02:17:35.497000 11182 consensus_queue.cc:260] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } }
I20260808 02:17:35.498811 11182 sys_catalog.cc:564] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:35.499536 11015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954: No bootstrap required, opened a new log
I20260808 02:17:35.497709 11206 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } } }
I20260808 02:17:35.501423 11206 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:17:35.505544 11217 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:35.505685 11217 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:35.503738 11106 master_runner.cc:186] Error getting master registration for 127.18.184.190:43825: OK, instance_id { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" instance_seqno: 1786155454938016 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:17:35.507370 11106 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:17:35.507961 11015 raft_consensus.cc:348] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } }
I20260808 02:17:35.508250 11015 raft_consensus.cc:729] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cecfd75219624e3483fdeff41cbcd954, State: Initialized, Role: FOLLOWER
I20260808 02:17:35.508615 11015 consensus_queue.cc:260] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } }
I20260808 02:17:35.509269 11220 sys_catalog.cc:455] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } } }
I20260808 02:17:35.509598 11220 sys_catalog.cc:458] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:35.510195 11015 sys_catalog.cc:564] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:35.510248 10987 raft_consensus.cc:482] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 02:17:35.510349 10987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:35.511914 10987 raft_consensus.cc:504] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } }
I20260808 02:17:35.513018 10987 leader_election.cc:288] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [CANDIDATE]: Term 1 election: Requested vote from peers 112998d8872c4605beac8084bcd68834 (127.18.184.189:43589), bff8bc1acf8f45c2a6557d67b3459c4a (127.18.184.188:35747)
I20260808 02:17:35.514590 11150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cecfd75219624e3483fdeff41cbcd954" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a"
I20260808 02:17:35.514863 11150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:35.515180 11068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cecfd75219624e3483fdeff41cbcd954" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "112998d8872c4605beac8084bcd68834"
I20260808 02:17:35.515468 11068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:35.516520 11150 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecfd75219624e3483fdeff41cbcd954 in term 1.
I20260808 02:17:35.517000 11068 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecfd75219624e3483fdeff41cbcd954 in term 1.
I20260808 02:17:35.518249 10962 leader_election.cc:302] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bff8bc1acf8f45c2a6557d67b3459c4a, cecfd75219624e3483fdeff41cbcd954; no voters:
W20260808 02:17:35.518806 11232 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:35.518886 11232 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:35.519433 11220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:35.519747 11220 raft_consensus.cc:686] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [term 1 LEADER]: Becoming Leader. State: Replica: cecfd75219624e3483fdeff41cbcd954, State: Running, Role: LEADER
I20260808 02:17:35.520171 11220 consensus_queue.cc:237] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } }
I20260808 02:17:35.522259 11220 sys_catalog.cc:455] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cecfd75219624e3483fdeff41cbcd954. Latest consensus state: current_term: 1 leader_uuid: "cecfd75219624e3483fdeff41cbcd954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } } }
I20260808 02:17:35.522451 11220 sys_catalog.cc:458] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:35.522917 11241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:35.524000 11241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:35.526304 10940 master_runner.cc:263] No leader master found from master cecfd75219624e3483fdeff41cbcd954
I20260808 02:17:35.526387 10940 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:17:35.532107 11150 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [term 1 FOLLOWER]: Refusing update from remote peer cecfd75219624e3483fdeff41cbcd954: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:35.533000 11235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:35.538120 11206 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cecfd75219624e3483fdeff41cbcd954. Latest consensus state: current_term: 1 leader_uuid: "cecfd75219624e3483fdeff41cbcd954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } } }
I20260808 02:17:35.538271 11206 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:35.539578 11068 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [term 1 FOLLOWER]: Refusing update from remote peer cecfd75219624e3483fdeff41cbcd954: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:35.540325 11235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:35.545547 11190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cecfd75219624e3483fdeff41cbcd954. Latest consensus state: current_term: 1 leader_uuid: "cecfd75219624e3483fdeff41cbcd954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } } }
I20260808 02:17:35.545701 11190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:35.552093 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.534520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36886 (local address 127.18.184.190:43825)
0808 02:17:35.534634 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.534638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.541566 (+ 6928us) server_negotiation.cc:407] Connection header received
0808 02:17:35.541808 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.541812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.541867 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.541921 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.543090 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.543897 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.545128 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.545503 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.545593 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.547788 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.547792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.547796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.547828 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.549992 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.550543 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.550547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.550549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.550642 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.550999 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.551002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.551004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.551252 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.551364 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.551770 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.551905 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:17:35.553037 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.533638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57090 (local address 127.18.184.188:35747)
0808 02:17:35.533779 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.533784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.541481 (+ 7697us) server_negotiation.cc:407] Connection header received
0808 02:17:35.542051 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.542055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.542120 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.542170 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.542979 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.543779 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.546404 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.546755 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.546836 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.548715 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.548720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.548723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.548755 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.550720 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.551271 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.551275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.551277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.551373 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.551728 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.551731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.551733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.551958 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.552056 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.552734 (+ 678us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.552865 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:17:35.558141 11235 sys_catalog.cc:455] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cecfd75219624e3483fdeff41cbcd954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } } }
I20260808 02:17:35.558310 11235 sys_catalog.cc:458] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:35.558643 11235 sys_catalog.cc:455] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cecfd75219624e3483fdeff41cbcd954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } } }
I20260808 02:17:35.558758 11235 sys_catalog.cc:458] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:35.559737 11206 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cecfd75219624e3483fdeff41cbcd954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } } }
I20260808 02:17:35.559875 11206 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:35.559789 11190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cecfd75219624e3483fdeff41cbcd954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cecfd75219624e3483fdeff41cbcd954" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 43825 } } peers { permanent_uuid: "112998d8872c4605beac8084bcd68834" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 43589 } } peers { permanent_uuid: "bff8bc1acf8f45c2a6557d67b3459c4a" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 35747 } } }
I20260808 02:17:35.559917 11190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:35.560333 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:35.535279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38976 (local address 127.18.184.189:43589)
0808 02:17:35.535426 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.535431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.541741 (+ 6310us) server_negotiation.cc:407] Connection header received
0808 02:17:35.543035 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.543038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.543096 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.543167 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.544231 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.544959 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.553450 (+ 8491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:35.553796 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:35.553897 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:35.555721 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:35.555724 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:35.555727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:35.555754 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:35.557521 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.558259 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.558265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.558266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.558360 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:35.558915 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:35.558918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:35.558944 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:35.559194 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:35.559314 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:35.559730 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 02:17:35.559851 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:17:35.560546 11242 mvcc.cc:205] Tried to move back new op lower bound from 7316092745850658816 to 7316092745815134208. Current Snapshot: MvccSnapshot[applied={T|T < 7316092745850658816}]
I20260808 02:17:35.560726 11246 mvcc.cc:205] Tried to move back new op lower bound from 7316092745850658816 to 7316092745815134208. Current Snapshot: MvccSnapshot[applied={T|T < 7316092745850658816}]
I20260808 02:17:35.561610 11248 mvcc.cc:205] Tried to move back new op lower bound from 7316092745850658816 to 7316092745815134208. Current Snapshot: MvccSnapshot[applied={T|T < 7316092745850658816}]
I20260808 02:17:35.566546 11241 catalog_manager.cc:1347] Generated new cluster ID: bd26a016454744b0aa5f0e24dfe34370
I20260808 02:17:35.566624 11241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:35.599025 11241 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:35.600689 11241 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:35.614346 11241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cecfd75219624e3483fdeff41cbcd954: Generated new TSK 0
I20260808 02:17:35.615537 11241 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:17:35.658788 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:35.654565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57094 (local address 127.18.184.188:35747)
0808 02:17:35.654717 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.654722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.657282 (+ 2560us) server_negotiation.cc:407] Connection header received
0808 02:17:35.657615 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.657620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.657687 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.657769 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.658607 (+ 838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57094: BlockingRecv error: recv got EOF from 127.0.0.1:57094 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 02:17:35.660885 11019 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:35.655592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36888 (local address 127.18.184.190:43825)
0808 02:17:35.655730 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.655735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.659295 (+ 3560us) server_negotiation.cc:407] Connection header received
0808 02:17:35.659975 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.659979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.660040 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.660099 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.660745 (+ 646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36888: BlockingRecv error: recv got EOF from 127.0.0.1:36888 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 02:17:35.661994 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:35.656376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38984 (local address 127.18.184.189:43589)
0808 02:17:35.656500 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.656506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.659674 (+ 3168us) server_negotiation.cc:407] Connection header received
0808 02:17:35.661200 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.661204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.661265 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.661324 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.661875 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38984: BlockingRecv error: recv got EOF from 127.0.0.1:38984 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 02:17:35.667383 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:35.665468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39000 (local address 127.18.184.189:43589)
0808 02:17:35.665595 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.665600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.666208 (+ 608us) server_negotiation.cc:407] Connection header received
0808 02:17:35.666504 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.666508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.666593 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.666655 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.667273 (+ 618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39000: BlockingRecv error: recv got EOF from 127.0.0.1:39000 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 02:17:35.673342 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:35.671498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39014 (local address 127.18.184.189:43589)
0808 02:17:35.671620 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.671625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.672248 (+ 623us) server_negotiation.cc:407] Connection header received
0808 02:17:35.672476 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.672479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.672537 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.672593 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.673209 (+ 616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39014: BlockingRecv error: recv got EOF from 127.0.0.1:39014 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 02:17:35.679903 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:35.678065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39024 (local address 127.18.184.189:43589)
0808 02:17:35.678218 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.678223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.678810 (+ 587us) server_negotiation.cc:407] Connection header received
0808 02:17:35.679021 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.679026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.679086 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.679149 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.679762 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39024: BlockingRecv error: recv got EOF from 127.0.0.1:39024 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 02:17:35.688867 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:35.686861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39038 (local address 127.18.184.189:43589)
0808 02:17:35.687027 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.687033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.687701 (+ 668us) server_negotiation.cc:407] Connection header received
0808 02:17:35.687951 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.687956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.688019 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.688080 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.688735 (+ 655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39038: BlockingRecv error: recv got EOF from 127.0.0.1:39038 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260808 02:17:35.702526 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:35.700136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39052 (local address 127.18.184.189:43589)
0808 02:17:35.700300 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.700306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.701072 (+ 766us) server_negotiation.cc:407] Connection header received
0808 02:17:35.701447 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.701452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.701519 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.701586 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.702391 (+ 805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39052: BlockingRecv error: recv got EOF from 127.0.0.1:39052 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 02:17:35.723791 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:35.721779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39062 (local address 127.18.184.189:43589)
0808 02:17:35.721924 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.721929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.722631 (+ 702us) server_negotiation.cc:407] Connection header received
0808 02:17:35.722863 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.722867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.722928 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.722986 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.723648 (+ 662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39062: BlockingRecv error: recv got EOF from 127.0.0.1:39062 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 02:17:35.760241 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:35.758765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39068 (local address 127.18.184.189:43589)
0808 02:17:35.758917 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.758922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.759202 (+ 280us) server_negotiation.cc:407] Connection header received
0808 02:17:35.759455 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.759458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.759517 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.759609 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.760105 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39068: BlockingRecv error: recv got EOF from 127.0.0.1:39068 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 02:17:35.828224 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:35.826713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39070 (local address 127.18.184.189:43589)
0808 02:17:35.826925 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.826931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.827098 (+ 167us) server_negotiation.cc:407] Connection header received
0808 02:17:35.827300 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.827304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.827361 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.827462 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.828098 (+ 636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39070: BlockingRecv error: recv got EOF from 127.0.0.1:39070 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260808 02:17:35.959826 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:35.958510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39084 (local address 127.18.184.189:43589)
0808 02:17:35.958668 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:35.958674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:35.958819 (+ 145us) server_negotiation.cc:407] Connection header received
0808 02:17:35.958991 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:35.958999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:35.959049 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:35.959141 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:35.959627 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39084: BlockingRecv error: recv got EOF from 127.0.0.1:39084 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 02:17:36.219416 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:36.218198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39092 (local address 127.18.184.189:43589)
0808 02:17:36.218331 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:36.218334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:36.218353 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:17:36.218555 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:36.218559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:36.218614 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:36.218697 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:36.219276 (+ 579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39092: BlockingRecv error: recv got EOF from 127.0.0.1:39092 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:17:36.496570 11201 catalog_manager.cc:1259] Loaded cluster ID: bd26a016454744b0aa5f0e24dfe34370
I20260808 02:17:36.496695 11201 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834: loading cluster ID for follower catalog manager: success
I20260808 02:17:36.498531 11201 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834: acquiring CA information for follower catalog manager: success
I20260808 02:17:36.499874 11201 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 112998d8872c4605beac8084bcd68834: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:17:36.507093 11217 catalog_manager.cc:1259] Loaded cluster ID: bd26a016454744b0aa5f0e24dfe34370
I20260808 02:17:36.507174 11217 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a: loading cluster ID for follower catalog manager: success
I20260808 02:17:36.508853 11217 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a: acquiring CA information for follower catalog manager: success
I20260808 02:17:36.510061 11217 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bff8bc1acf8f45c2a6557d67b3459c4a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:17:36.738030 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:36.733781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.18.184.189:43589)
0808 02:17:36.734024 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:36.734028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:36.734045 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:17:36.734267 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:36.734274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:36.734502 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:36.734618 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:36.734961 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:36.735602 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:36.736513 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:36.736787 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:36.736880 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:36.737196 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:36.737653 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:36.737749 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:36.737839 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 02:17:36.737894 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":92,"threads_started":1}
I20260808 02:17:36.745339 11334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:36.741450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.18.184.188:35747)
0808 02:17:36.741713 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:36.741717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:36.741734 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:17:36.741826 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:36.741828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:36.741946 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:36.742055 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:36.742257 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:36.742787 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:36.743640 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:36.743895 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:36.743986 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:36.744358 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:36.744834 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:36.744934 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:36.745042 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 02:17:36.745106 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":117,"threads_started":1}
I20260808 02:17:36.753363 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:36.749588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39106 (local address 127.18.184.189:43589)
0808 02:17:36.749746 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:36.749750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:36.750012 (+ 262us) server_negotiation.cc:407] Connection header received
0808 02:17:36.750199 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:36.750202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:36.750316 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:36.750402 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:36.750628 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:36.751245 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:36.752216 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:36.752446 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:36.752554 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:36.752821 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:36.752894 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:36.752973 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:36.753097 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 02:17:36.753181 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:17:36.754297 11334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:36.749092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57114 (local address 127.18.184.188:35747)
0808 02:17:36.749232 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:36.749237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:36.749406 (+ 169us) server_negotiation.cc:407] Connection header received
0808 02:17:36.749586 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:36.749589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:36.749676 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:36.749734 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:36.750298 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:36.751123 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:36.752629 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:36.753003 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:36.753099 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:36.753717 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:36.753824 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:36.753897 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:36.754091 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 02:17:36.754160 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:17:36.754767 11342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:36.748966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36900 (local address 127.18.184.190:43825)
0808 02:17:36.749304 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:36.749309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:36.749329 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:17:36.749556 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:36.749559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:36.749745 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:36.749902 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:36.750538 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:36.751355 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:36.752639 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:36.753023 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:36.753124 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:36.753792 (+ 668us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:36.754199 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:36.754299 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:36.754464 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 02:17:36.754533 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":94,"threads_started":1}
I20260808 02:17:36.758390 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 10940
I20260808 02:17:36.767992 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 11022
I20260808 02:17:36.777105 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 11106
2026-08-08T02:17:36Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3298 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:36 dist-test-slave-9gcw krb5kdc[11346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:36 dist-test-slave-9gcw krb5kdc[11346](info): set up 2 sockets
Aug 08 02:17:36 dist-test-slave-9gcw krb5kdc[11346](info): commencing operation
krb5kdc: starting...
W20260808 02:17:37.908277 19170 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/test-user.keytab.
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.184.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
Entry for principal kudu/127.18.184.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab.
I20260808 02:17:38.048900 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:35069
--webserver_interface=127.18.184.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:35069,127.18.184.189:34895,127.18.184.188:46131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:38.163502 11361 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:17:38.163851 11361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:38.163921 11361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:38.164004 11361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:38.168126 11361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:38.168228 11361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:38.168269 11361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:38.168323 11361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:38.168359 11361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:38.173853 11361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:35069,127.18.184.189:34895,127.18.184.188:46131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.190.keytab
--principal=kudu/127.18.184.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.190:35069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.184.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:38.175002 11361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:38.176082 11361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:38.184782 11367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:38.184800 11370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:38.185029 11361 server_base.cc:1034] running on GCE node
W20260808 02:17:38.184800 11368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:38.185683 11361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:38.186793 11361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:38.187979 11361 hybrid_clock.cc:648] HybridClock initialized: now 1786155458187952 us; error 41 us; skew 500 ppm
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:38.195998 11361 init.cc:377] Logged in from keytab as kudu/127.18.184.190@KRBTEST.COM (short username kudu)
I20260808 02:17:38.196969 11361 webserver.cc:460] Webserver started at http://127.18.184.190:42893/ using document root <none> and password file <none>
I20260808 02:17:38.197600 11361 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:38.197669 11361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:38.197844 11361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:38.199895 11361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/data/instance:
uuid: "c4e24a82498e40e5b7866ab09c5f2d29"
format_stamp: "Formatted at 2026-08-08 02:17:38 on dist-test-slave-9gcw"
server_key: "bba46ced2631819a2e12770b6927f8db"
server_key_iv: "53eaa483a2c016e8e09888255bb5d3ee"
server_key_version: "encryptionkey@0"
I20260808 02:17:38.200436 11361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/wal/instance:
uuid: "c4e24a82498e40e5b7866ab09c5f2d29"
format_stamp: "Formatted at 2026-08-08 02:17:38 on dist-test-slave-9gcw"
server_key: "bba46ced2631819a2e12770b6927f8db"
server_key_iv: "53eaa483a2c016e8e09888255bb5d3ee"
server_key_version: "encryptionkey@0"
I20260808 02:17:38.204259 11361 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 02:17:38.206821 11378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:38.208022 11361 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:17:38.208140 11361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/wal
uuid: "c4e24a82498e40e5b7866ab09c5f2d29"
format_stamp: "Formatted at 2026-08-08 02:17:38 on dist-test-slave-9gcw"
server_key: "bba46ced2631819a2e12770b6927f8db"
server_key_iv: "53eaa483a2c016e8e09888255bb5d3ee"
server_key_version: "encryptionkey@0"
I20260808 02:17:38.208231 11361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:38.227388 11361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:38.230760 11361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:38.230935 11361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:38.238236 11361 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.190:35069
I20260808 02:17:38.238551 11433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.190:35069 every 8 connection(s)
I20260808 02:17:38.239296 11361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:38.242194 11434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:38.246025 11434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:38.246675 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 11361
I20260808 02:17:38.246822 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-0/wal/instance
I20260808 02:17:38.247104 19170 external_mini_cluster.cc:1428] Setting key 918e46c70c1babb004385d21430dd2f1
WARNING: no policy specified for kudu/127.18.184.189@KRBTEST.COM; defaulting to no policy
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.189@KRBTEST.COM" created.
I20260808 02:17:38.261761 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.248414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:35069 (local address 127.0.0.1:60230)
0808 02:17:38.248991 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.249061 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:38.250060 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:38.250753 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:38.250759 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:38.251234 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:38.251487 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.251500 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.252641 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.252644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.253444 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.253453 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.254076 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.254081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.254188 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.254864 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:38.254880 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:38.256415 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:38.258358 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.258367 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.258375 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.258658 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.259023 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.259030 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.259031 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.259210 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.259894 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:38.259900 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:38.260048 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 02:17:38.260232 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 02:17:38.260586 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":119,"threads_started":1}
I20260808 02:17:38.261761 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.249283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60230 (local address 127.18.184.190:35069)
0808 02:17:38.249570 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.249582 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.249627 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:17:38.250376 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.250381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.250564 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.250753 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.251640 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.252516 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.253558 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.253922 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.254097 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.256567 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.256573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.256582 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.256605 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.258225 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.258777 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.258782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.258783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.258904 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.259347 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.259351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.259354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.259755 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.259928 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.260362 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.260725 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":192,"thread_start_us":90,"threads_started":1}
I20260808 02:17:38.264374 11434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:38.265314 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.264961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:34895 (local address 127.0.0.1:60578)
0808 02:17:38.265095 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.265224 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:17:38.265581 11381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.184.189:34895: connect: Connection refused (error 111)
I20260808 02:17:38.266467 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.266238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:34895 (local address 127.0.0.1:60588)
0808 02:17:38.266351 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.266411 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 02:17:38.266633 11434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.189:34895: Network error: Client connection negotiation failed: client connection to 127.18.184.189:34895: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.189.keytab.
Entry for principal kudu/127.18.184.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.189.keytab.
I20260808 02:17:38.280643 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.189:34895
--webserver_interface=127.18.184.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.189.keytab
--principal=kudu/127.18.184.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:35069,127.18.184.189:34895,127.18.184.188:46131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:17:38.315902 11434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } attempt: 1
I20260808 02:17:38.316902 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.316566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:34895 (local address 127.0.0.1:60590)
0808 02:17:38.316716 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.316816 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 02:17:38.317685 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.317461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:34895 (local address 127.0.0.1:60594)
0808 02:17:38.317574 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.317625 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 02:17:38.317843 11434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.189:34895: Network error: Client connection negotiation failed: client connection to 127.18.184.189:34895: connect: Connection refused (error 111)
I20260808 02:17:38.386024 11434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } attempt: 2
I20260808 02:17:38.387085 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.386638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:34895 (local address 127.0.0.1:60598)
0808 02:17:38.386895 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.386973 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260808 02:17:38.387858 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.387622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:34895 (local address 127.0.0.1:60612)
0808 02:17:38.387739 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.387794 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.189:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 02:17:38.388017 11434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.189:34895: Network error: Client connection negotiation failed: client connection to 127.18.184.189:34895: connect: Connection refused (error 111)
W20260808 02:17:38.402501 11442 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:17:38.402910 11442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:38.402979 11442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:38.403069 11442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:38.407277 11442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:38.407378 11442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:38.407423 11442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:38.407465 11442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:38.407524 11442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:38.413409 11442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:35069,127.18.184.189:34895,127.18.184.188:46131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.189.keytab
--principal=kudu/127.18.184.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.189:34895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.184.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:38.414804 11442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:38.415957 11442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:38.425547 11447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:38.425601 11450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:38.425578 11448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:38.426044 11442 server_base.cc:1034] running on GCE node
I20260808 02:17:38.426627 11442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:38.427945 11442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:38.429132 11442 hybrid_clock.cc:648] HybridClock initialized: now 1786155458429115 us; error 40 us; skew 500 ppm
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:38.432544 11442 init.cc:377] Logged in from keytab as kudu/127.18.184.189@KRBTEST.COM (short username kudu)
I20260808 02:17:38.433756 11442 webserver.cc:460] Webserver started at http://127.18.184.189:34181/ using document root <none> and password file <none>
I20260808 02:17:38.434396 11442 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:38.434465 11442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:38.434739 11442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:38.437063 11442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/data/instance:
uuid: "0832c5eea1b24f1eb656c019a0e134bf"
format_stamp: "Formatted at 2026-08-08 02:17:38 on dist-test-slave-9gcw"
server_key: "9c72d7060e209a3458c1f4c39204f0a5"
server_key_iv: "dbbca12626380ef26f54f2d96c23cd62"
server_key_version: "encryptionkey@0"
I20260808 02:17:38.437767 11442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/wal/instance:
uuid: "0832c5eea1b24f1eb656c019a0e134bf"
format_stamp: "Formatted at 2026-08-08 02:17:38 on dist-test-slave-9gcw"
server_key: "9c72d7060e209a3458c1f4c39204f0a5"
server_key_iv: "dbbca12626380ef26f54f2d96c23cd62"
server_key_version: "encryptionkey@0"
I20260808 02:17:38.442054 11442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:17:38.444770 11458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:38.446218 11442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:38.446393 11442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/wal
uuid: "0832c5eea1b24f1eb656c019a0e134bf"
format_stamp: "Formatted at 2026-08-08 02:17:38 on dist-test-slave-9gcw"
server_key: "9c72d7060e209a3458c1f4c39204f0a5"
server_key_iv: "dbbca12626380ef26f54f2d96c23cd62"
server_key_version: "encryptionkey@0"
I20260808 02:17:38.446521 11442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:38.465678 11442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:38.469055 11442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:38.469324 11442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:38.476737 11442 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.189:34895
I20260808 02:17:38.477116 11518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.189:34895 every 8 connection(s)
I20260808 02:17:38.477849 11442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/data/info.pb
I20260808 02:17:38.478888 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 11442
I20260808 02:17:38.479036 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-1/wal/instance
I20260808 02:17:38.479167 11434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } attempt: 3
I20260808 02:17:38.479348 19170 external_mini_cluster.cc:1428] Setting key b658fd2c240ab01e72ebdee9b82eda8f
I20260808 02:17:38.481289 11519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:38.486492 11519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.184.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.184.188@KRBTEST.COM" created.
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.189@KRBTEST.COM
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.189@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:17:38.498338 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.479774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:34895 (local address 127.0.0.1:60624)
0808 02:17:38.479932 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.479951 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:38.480059 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:38.483599 (+ 3540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:38.483606 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:38.483858 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:38.484043 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.484055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.485986 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.485994 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.486993 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.487012 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.487991 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.487998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.488123 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.488719 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:38.488741 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:38.493328 (+ 4587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:38.495932 (+ 2604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.495936 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.495939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.496248 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.496665 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.496668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.496669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.496785 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.497694 (+ 909us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:38.497700 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:38.497823 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 02:17:38.497948 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 02:17:38.498104 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 02:17:38.499675 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.488767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.18.184.190:35069)
0808 02:17:38.488909 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.488913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.489340 (+ 427us) server_negotiation.cc:407] Connection header received
0808 02:17:38.489535 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.489538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.489590 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.489683 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.490418 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.491488 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.492437 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.492765 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.492876 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.496033 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.496037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.496040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.496065 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.497663 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.498242 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.498246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.498248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.498331 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.498671 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.498674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.498676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.498882 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.498977 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.499405 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.499525 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:17:38.499928 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.488587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:35069 (local address 127.0.0.1:60244)
0808 02:17:38.489200 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.489267 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:38.489440 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:38.489740 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:38.489746 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:38.490141 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:38.490285 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.490291 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.491625 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.491630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.492307 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.492317 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.493018 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.493025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.493162 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.493855 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:38.493868 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:38.495884 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:38.497800 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.497810 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.497816 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.498127 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.498454 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.498459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.498461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.498560 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.499072 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:38.499078 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:38.499187 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 02:17:38.499331 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 02:17:38.499525 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":134,"threads_started":1}
I20260808 02:17:38.499945 11521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.481016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60624 (local address 127.18.184.189:34895)
0808 02:17:38.481617 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.481637 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.481683 (+ 46us) server_negotiation.cc:407] Connection header received
0808 02:17:38.482747 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.482772 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.483217 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.483620 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.484234 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.485812 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.487147 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.487832 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.487991 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.493521 (+ 5530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.493531 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.493548 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.493589 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.495804 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.496398 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.496404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.496409 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.496570 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.496914 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.496921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.496925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.497569 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.497715 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.498129 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.498663 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":162,"threads_started":1}
I20260808 02:17:38.501870 11519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:38.502770 11434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:38.503465 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.503210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:47900)
0808 02:17:38.503329 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.503390 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 02:17:38.504091 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.503856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:47904)
0808 02:17:38.503992 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.504044 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260808 02:17:38.504271 11434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:46131: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.189@KRBTEST.COM for kudu/127.18.184.189@KRBTEST.COM
I20260808 02:17:38.511226 11521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.502492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60634 (local address 127.18.184.189:34895)
0808 02:17:38.502833 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.502839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.502852 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:17:38.503156 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.503161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.503219 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.503325 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.503870 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.504587 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.505397 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.505738 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.505824 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.507839 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.507842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.507844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.507878 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.509353 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.509886 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.509889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.509890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.509969 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.510340 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.510343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.510345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.510552 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.510648 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.510953 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.511112 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270}
I20260808 02:17:38.511227 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.502449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:34895 (local address 127.0.0.1:60634)
0808 02:17:38.502561 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.502575 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:38.502675 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:38.503456 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:38.503459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:38.503642 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:38.503749 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.503755 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.504687 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.504690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.505303 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.505313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.505839 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.505842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.505915 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.506412 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:38.506432 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:38.507696 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:38.509475 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.509478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.509481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.509741 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.510079 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.510087 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.510088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.510215 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.510645 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:38.510647 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:38.510757 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 02:17:38.510862 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 02:17:38.511041 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.184.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.188.keytab.
Entry for principal kudu/127.18.184.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.188.keytab.
I20260808 02:17:38.512828 11519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:38.513325 19170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmhslVu/build/debug/bin/kudu
/tmp/dist-test-taskmhslVu/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.188:46131
--webserver_interface=127.18.184.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.188.keytab
--principal=kudu/127.18.184.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.184.148:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.184.190:35069,127.18.184.189:34895,127.18.184.188:46131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:17:38.513787 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.513426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:47918)
0808 02:17:38.513641 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.513721 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
W20260808 02:17:38.514062 11461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
I20260808 02:17:38.515230 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.514959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:47920)
0808 02:17:38.515084 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.515143 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 02:17:38.515506 11519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:46131: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
I20260808 02:17:38.535470 11434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } attempt: 1
I20260808 02:17:38.536433 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.536055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:47930)
0808 02:17:38.536253 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.536335 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 02:17:38.537362 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.537065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:47934)
0808 02:17:38.537244 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.537299 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 02:17:38.537544 11434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:46131: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
I20260808 02:17:38.564834 11519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } attempt: 1
I20260808 02:17:38.565836 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.565475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:47950)
0808 02:17:38.565638 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.565704 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:17:38.566639 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.566411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:47962)
0808 02:17:38.566530 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.566581 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 02:17:38.566766 11519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:46131: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
I20260808 02:17:38.612718 11434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } attempt: 2
I20260808 02:17:38.613691 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.613381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:47974)
0808 02:17:38.613543 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.613608 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:17:38.614466 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.614248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:47980)
0808 02:17:38.614354 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.614405 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 02:17:38.614594 11434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:46131: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
W20260808 02:17:38.631641 11526 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:17:38.632025 11526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:38.632097 11526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:38.632180 11526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 02:17:38.634931 11519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } attempt: 2
I20260808 02:17:38.635819 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.635509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:47992)
0808 02:17:38.635661 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.635732 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 02:17:38.636593 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.636374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:48008)
0808 02:17:38.636485 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.636534 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 02:17:38.636720 11519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:46131: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
W20260808 02:17:38.636724 11526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:38.636812 11526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:38.636858 11526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:38.636911 11526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:38.636950 11526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:38.642611 11526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.184.148:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.184.190:35069,127.18.184.189:34895,127.18.184.188:46131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/krb5kdc/kudu_127.18.184.188.keytab
--principal=kudu/127.18.184.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.184.188:46131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.184.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:38.643832 11526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:38.644905 11526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:17:38.654467 11526 server_base.cc:1034] running on GCE node
W20260808 02:17:38.654438 11532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:38.654407 11531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:38.654407 11534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:38.655400 11526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:38.656662 11526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:38.657845 11526 hybrid_clock.cc:648] HybridClock initialized: now 1786155458657807 us; error 52 us; skew 500 ppm
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:38.661121 11526 init.cc:377] Logged in from keytab as kudu/127.18.184.188@KRBTEST.COM (short username kudu)
I20260808 02:17:38.662153 11526 webserver.cc:460] Webserver started at http://127.18.184.188:33001/ using document root <none> and password file <none>
I20260808 02:17:38.662882 11526 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:38.662961 11526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:38.663153 11526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:38.665562 11526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/data/instance:
uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0"
format_stamp: "Formatted at 2026-08-08 02:17:38 on dist-test-slave-9gcw"
server_key: "f8b31a34a9640a3a1477d09527167d38"
server_key_iv: "599fff15585b8c104a0f04c2c15b741d"
server_key_version: "encryptionkey@0"
I20260808 02:17:38.666158 11526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/wal/instance:
uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0"
format_stamp: "Formatted at 2026-08-08 02:17:38 on dist-test-slave-9gcw"
server_key: "f8b31a34a9640a3a1477d09527167d38"
server_key_iv: "599fff15585b8c104a0f04c2c15b741d"
server_key_version: "encryptionkey@0"
I20260808 02:17:38.670492 11526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:17:38.673310 11541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:38.674728 11526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:38.674896 11526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/data,/tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/wal
uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0"
format_stamp: "Formatted at 2026-08-08 02:17:38 on dist-test-slave-9gcw"
server_key: "f8b31a34a9640a3a1477d09527167d38"
server_key_iv: "599fff15585b8c104a0f04c2c15b741d"
server_key_version: "encryptionkey@0"
I20260808 02:17:38.675017 11526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:38.713748 11434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } attempt: 3
I20260808 02:17:38.714685 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.714355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:48016)
0808 02:17:38.714519 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.714589 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 02:17:38.715484 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.715238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:48028)
0808 02:17:38.715366 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.715420 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 02:17:38.715626 11434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.184.188:46131: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
I20260808 02:17:38.717638 11526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:38.721110 11526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:38.721355 11526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:38.727891 11519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } attempt: 3
I20260808 02:17:38.728744 11526 rpc_server.cc:307] RPC server started. Bound to: 127.18.184.188:46131
I20260808 02:17:38.728811 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.728490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:48030)
0808 02:17:38.728612 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.728708 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.184.188:46131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 02:17:38.729332 11599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.184.188:46131 every 8 connection(s)
I20260808 02:17:38.729965 11526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/data/info.pb
I20260808 02:17:38.731967 19170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmhslVu/build/debug/bin/kudu as pid 11526
I20260808 02:17:38.732110 19170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmhslVu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155161841348-19170-0/minicluster-data/master-2/wal/instance
I20260808 02:17:38.732456 19170 external_mini_cluster.cc:1428] Setting key d299301e834e20103e5dfabf0d3c5712
I20260808 02:17:38.733378 11600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.189@KRBTEST.COM for kudu/127.18.184.188@KRBTEST.COM
I20260808 02:17:38.738214 11600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.188@KRBTEST.COM
I20260808 02:17:38.743112 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.729374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:48038)
0808 02:17:38.729587 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.729602 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:38.729802 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:38.732276 (+ 2474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:38.732280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:38.732508 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:38.732654 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.732665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.734314 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.734318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.735132 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.735148 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.735889 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.735894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.735999 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.736501 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:38.736518 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:38.738185 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:38.740731 (+ 2546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.740736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.740739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.740963 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.741442 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.741446 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.741451 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.741558 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.742508 (+ 950us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:38.742511 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:38.742634 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 02:17:38.742754 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 02:17:38.742915 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260808 02:17:38.744270 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.730313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48038 (local address 127.18.184.188:46131)
0808 02:17:38.730758 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.730777 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.730820 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:17:38.731460 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.731476 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.731883 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.732333 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.732814 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.734174 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.735277 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.735762 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.735879 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.738386 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.738391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.738404 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.738430 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.740588 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.741087 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.741097 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.741104 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.741294 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.741738 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.741744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.741749 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.742331 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.742539 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.742924 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.743235 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":138,"threads_started":1}
I20260808 02:17:38.747434 11605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.733074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48052 (local address 127.18.184.188:46131)
0808 02:17:38.736612 (+ 3538us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.736619 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.736630 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:17:38.736657 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.736659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.736691 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.736776 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.737807 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.738598 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.739769 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.740080 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.740194 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.743610 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.743614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.743617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.743643 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.745290 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.745844 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.745847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.745850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.745955 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.746277 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.746280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.746282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.746523 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.746694 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.747044 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.747197 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3469,"spinlock_wait_cycles":768,"thread_start_us":155,"threads_started":1}
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.188@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:17:38.749060 11519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf: Bootstrap starting.
I20260808 02:17:38.752293 11519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:38.752907 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.741185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60246 (local address 127.18.184.190:35069)
0808 02:17:38.741321 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.741331 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.741343 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:17:38.741380 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.741382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.741419 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.741507 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.742147 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.742850 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.744350 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.745449 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.745558 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.749212 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.749224 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.749227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.749260 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.750960 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.751467 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.751471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.751472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.751557 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.751844 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.751849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.751851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.752088 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.752182 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.752632 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.752749 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:17:38.752988 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.740299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.190:35069 (local address 127.0.0.1:60246)
0808 02:17:38.740882 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.740935 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:38.741043 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:38.741542 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:38.741547 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:38.741887 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:38.742018 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.742024 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.743060 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.743066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.744205 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.744215 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.745687 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.745690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.745803 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.747373 (+ 1570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:38.747386 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:38.749036 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:38.751078 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.751087 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.751093 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.751330 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.751656 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.751660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.751662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.751752 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.752233 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:38.752240 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:38.752393 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 02:17:38.752573 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 02:17:38.752781 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":2,"thread_start_us":107,"threads_started":1}
I20260808 02:17:38.753343 11519 log.cc:824] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:38.754351 11600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.190@KRBTEST.COM
I20260808 02:17:38.756403 11519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf: No bootstrap required, opened a new log
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.188@KRBTEST.COM for kudu/127.18.184.189@KRBTEST.COM
I20260808 02:17:38.761092 11519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } }
I20260808 02:17:38.761595 11519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0832c5eea1b24f1eb656c019a0e134bf, State: Initialized, Role: FOLLOWER
I20260808 02:17:38.761891 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.749612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60262 (local address 127.18.184.190:35069)
0808 02:17:38.749950 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.749962 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.749973 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:17:38.750003 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.750004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.750040 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.750157 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.751130 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.751936 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.753003 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.753408 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.753505 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.756381 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.756386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.756389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.756417 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.759111 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.759616 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.759620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.759622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.759727 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.760115 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.760119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.760121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.760379 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.760591 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.761516 (+ 925us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.761692 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":110,"threads_started":1}
I20260808 02:17:38.761986 11519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } }
I20260808 02:17:38.762843 11610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } } }
I20260808 02:17:38.763000 11610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:38.763839 11521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.755007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.18.184.189:34895)
0808 02:17:38.755150 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.755156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.755186 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:17:38.755239 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.755244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.755301 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.755390 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.755972 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.756667 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.757702 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.758035 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.758153 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.760171 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.760174 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.760182 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.760205 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.761713 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.762228 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.762232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.762234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.762339 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.762715 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.762719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.762721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.763001 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.763088 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.763500 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.763643 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:17:38.763665 11519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:38.764045 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.754934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.189:34895 (local address 127.0.0.1:60638)
0808 02:17:38.755034 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.755061 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:38.755142 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:38.755439 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:38.755443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:38.755647 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:38.755779 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.755787 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.756781 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.756785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.757579 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.757592 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.758150 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.758153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.758228 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.758640 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:38.758653 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:38.760029 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:38.761849 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.761852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.761854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.762114 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.762475 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.762478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.762480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.762581 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.763180 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:38.763183 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:38.763291 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 02:17:38.763702 (+ 411us) client_negotiation.cc:240] Negotiation successful
0808 02:17:38.763855 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 02:17:38.764791 11600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } has no permanent_uuid. Determining permanent_uuid...
W20260808 02:17:38.770087 11623 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:38.770187 11623 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.184.189@KRBTEST.COM
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.188@KRBTEST.COM for kudu/127.18.184.188@KRBTEST.COM
I20260808 02:17:38.768548 11442 master_runner.cc:186] Error getting master registration for 127.18.184.190:35069: OK, instance_id { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" instance_seqno: 1786155458236193 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:17:38.773468 11442 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:17:38.774811 11605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.765532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48060 (local address 127.18.184.188:46131)
0808 02:17:38.765666 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.765670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.765682 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:17:38.765714 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.765717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.765758 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.765838 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.766337 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.767033 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.768315 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.768614 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.768704 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.771478 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.771481 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.771483 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.771504 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.773054 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.773595 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.773599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.773600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.773680 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.773946 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.773951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.773953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.774186 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.774294 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.774532 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.774689 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":56}
I20260808 02:17:38.774811 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.765320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:48060)
0808 02:17:38.765441 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.765458 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:38.765542 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:38.765878 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:38.765881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:38.766078 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:38.766203 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.766216 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.767150 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.767153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.768179 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.768192 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.768809 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.768812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.768883 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.769392 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:38.769406 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:38.771338 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:38.773190 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.773194 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.773197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.773453 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.773778 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.773781 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.773783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.773864 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.774281 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:38.774284 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:38.774378 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 02:17:38.774477 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 02:17:38.774627 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 02:17:38.778569 11600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0: Bootstrap starting.
I20260808 02:17:38.780128 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.763249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.18.184.189:34895)
0808 02:17:38.763567 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.763577 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.763591 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:17:38.763688 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.763691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.763742 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.763831 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.764714 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.765524 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.766622 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.767016 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.767132 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.772085 (+ 4953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.772089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.772092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.772124 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.777895 (+ 5771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.778443 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.778448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.778450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.778567 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.779038 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.779041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.779043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.779269 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.779498 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.779717 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.779879 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":103,"threads_started":1}
I20260808 02:17:38.781854 11600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:38.782829 11600 log.cc:824] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:38.784360 19170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:38.785842 11600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0: No bootstrap required, opened a new log
I20260808 02:17:38.785660 11406 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c4e24a82498e40e5b7866ab09c5f2d29"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60262
I20260808 02:17:38.790302 11600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } }
I20260808 02:17:38.790597 11600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f8d70d26d194b1e8d1ccb54e7695bf0, State: Initialized, Role: FOLLOWER
I20260808 02:17:38.790977 11600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } }
I20260808 02:17:38.791574 11627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } } }
I20260808 02:17:38.791749 11627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:38.792531 11600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 02:17:38.798852 11641 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:38.798955 11641 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:38.797235 11526 master_runner.cc:186] Error getting master registration for 127.18.184.190:35069: OK, instance_id { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" instance_seqno: 1786155458236193 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:17:38.800307 11526 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:17:38.879801 11434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } attempt: 4
Aug 08 02:17:38 dist-test-slave-9gcw krb5kdc[11346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.184.190@KRBTEST.COM for kudu/127.18.184.188@KRBTEST.COM
I20260808 02:17:38.890898 11605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.880515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48062 (local address 127.18.184.188:46131)
0808 02:17:38.880671 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.880676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.880766 (+ 90us) server_negotiation.cc:407] Connection header received
0808 02:17:38.880947 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.880952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.881010 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.881096 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.881619 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.882506 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.883590 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.883988 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.884092 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.887173 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.887177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.887180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.887214 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.888898 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.889484 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.889489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.889491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.889582 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.889914 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.889918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.889920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.890146 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.890270 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.890575 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.890729 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:17:38.890898 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.880464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.184.188:46131 (local address 127.0.0.1:48062)
0808 02:17:38.880692 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:38.880710 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:38.880824 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:38.881118 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:38.881122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:38.881379 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:38.881479 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.881486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.882632 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.882636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.883444 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:38.883456 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.884100 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.884104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:38.884206 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.884832 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:38.884849 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:38.887012 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:38.889032 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.889040 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.889042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.889385 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.889701 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:38.889707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:38.889709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:38.889808 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:38.890257 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:38.890261 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:38.890383 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 02:17:38.890488 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 02:17:38.890694 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260808 02:17:38.894750 11434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29: Bootstrap starting.
I20260808 02:17:38.897411 11434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:38.898238 11434 log.cc:824] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:38.900344 11434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29: No bootstrap required, opened a new log
I20260808 02:17:38.903968 11434 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } }
I20260808 02:17:38.904222 11434 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4e24a82498e40e5b7866ab09c5f2d29, State: Initialized, Role: FOLLOWER
I20260808 02:17:38.904589 11434 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } }
I20260808 02:17:38.905267 11646 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } } }
I20260808 02:17:38.905558 11646 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:38.906158 11434 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:38.906203 11406 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 02:17:38.906303 11406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:38.907845 11406 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } }
I20260808 02:17:38.909775 11490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0832c5eea1b24f1eb656c019a0e134bf"
I20260808 02:17:38.909909 11571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0"
I20260808 02:17:38.910049 11490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:38.910158 11571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:38.911512 11571 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4e24a82498e40e5b7866ab09c5f2d29 in term 1.
I20260808 02:17:38.911509 11490 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4e24a82498e40e5b7866ab09c5f2d29 in term 1.
I20260808 02:17:38.908933 11406 leader_election.cc:288] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [CANDIDATE]: Term 1 election: Requested vote from peers 0832c5eea1b24f1eb656c019a0e134bf (127.18.184.189:34895), 3f8d70d26d194b1e8d1ccb54e7695bf0 (127.18.184.188:46131)
I20260808 02:17:38.912324 11381 leader_election.cc:302] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0832c5eea1b24f1eb656c019a0e134bf, c4e24a82498e40e5b7866ab09c5f2d29; no voters:
I20260808 02:17:38.914270 11646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:38.914592 11646 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [term 1 LEADER]: Becoming Leader. State: Replica: c4e24a82498e40e5b7866ab09c5f2d29, State: Running, Role: LEADER
I20260808 02:17:38.914992 11646 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } }
W20260808 02:17:38.915302 11657 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:38.915400 11657 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:38.916420 11361 master_runner.cc:263] No leader master found from master c4e24a82498e40e5b7866ab09c5f2d29
I20260808 02:17:38.916492 11361 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:17:38.917296 11646 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e24a82498e40e5b7866ab09c5f2d29. Latest consensus state: current_term: 1 leader_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } } }
I20260808 02:17:38.917490 11646 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:38.920300 11665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:38.922247 11665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:38.964035 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.919451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60278 (local address 127.18.184.190:35069)
0808 02:17:38.919576 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.919580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.929094 (+ 9514us) server_negotiation.cc:407] Connection header received
0808 02:17:38.931690 (+ 2596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.931694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.931755 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.931816 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.937806 (+ 5990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.938730 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.942855 (+ 4125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.943336 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.943425 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.951952 (+ 8527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.951956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.951960 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.951994 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.953996 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.957934 (+ 3938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.957938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.957941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.958053 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.959412 (+ 1359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.959417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.959419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.959697 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.959826 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.963640 (+ 3814us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.963836 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 02:17:38.966109 11605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.918930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48076 (local address 127.18.184.188:46131)
0808 02:17:38.919051 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.919056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.928796 (+ 9740us) server_negotiation.cc:407] Connection header received
0808 02:17:38.929356 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.929360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.930298 (+ 938us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.930413 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.935497 (+ 5084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.936400 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.941586 (+ 5186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.942026 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.942140 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.949651 (+ 7511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.949656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.949659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.949699 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.951731 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.958435 (+ 6704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.958440 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.958442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.958550 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.960229 (+ 1679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.960234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.960236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.960562 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.960707 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.965749 (+ 5042us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.965913 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:17:38.968737 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:38.918172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60654 (local address 127.18.184.189:34895)
0808 02:17:38.918315 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:38.918320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:38.926180 (+ 7860us) server_negotiation.cc:407] Connection header received
0808 02:17:38.926447 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:38.926453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:38.926523 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:38.926598 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:38.932982 (+ 6384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.934029 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.940057 (+ 6028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:38.940542 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:38.940683 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:38.955188 (+ 14505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:38.955193 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:38.955196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:38.955231 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:38.957332 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.958909 (+ 1577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.958914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.958916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.959011 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:38.962807 (+ 3796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:38.962811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:38.962813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:38.963071 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:38.963257 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:38.968393 (+ 5136us) server_negotiation.cc:299] Negotiation successful
0808 02:17:38.968557 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 02:17:38.971104 11464 tablet.cc:2369] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:17:38.972821 11547 tablet.cc:2369] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:17:38.973872 11571 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [term 1 FOLLOWER]: Refusing update from remote peer c4e24a82498e40e5b7866ab09c5f2d29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:38.974655 11663 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:38.974949 11490 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [term 1 FOLLOWER]: Refusing update from remote peer c4e24a82498e40e5b7866ab09c5f2d29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 02:17:38.982664 11384 tablet.cc:2369] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:17:38.984139 11663 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:38.988967 11610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e24a82498e40e5b7866ab09c5f2d29. Latest consensus state: current_term: 1 leader_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } } }
I20260808 02:17:38.989135 11610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:38.991607 11627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e24a82498e40e5b7866ab09c5f2d29. Latest consensus state: current_term: 1 leader_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } } }
I20260808 02:17:38.991786 11627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:38.993353 11663 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } } }
I20260808 02:17:38.993494 11663 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:38.993607 11670 mvcc.cc:205] Tried to move back new op lower bound from 7316092759907074048 to 7316092759720706048. Current Snapshot: MvccSnapshot[applied={T|T < 7316092759907074048}]
I20260808 02:17:38.995589 11627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } } }
I20260808 02:17:38.995718 11627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:38.999053 11665 catalog_manager.cc:1347] Generated new cluster ID: 64fe63929fde4eb898bd4b1d74d28e12
I20260808 02:17:38.999140 11665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:39.002688 11646 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } } }
I20260808 02:17:39.002832 11646 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:39.004251 11610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4e24a82498e40e5b7866ab09c5f2d29" member_type: VOTER last_known_addr { host: "127.18.184.190" port: 35069 } } peers { permanent_uuid: "0832c5eea1b24f1eb656c019a0e134bf" member_type: VOTER last_known_addr { host: "127.18.184.189" port: 34895 } } peers { permanent_uuid: "3f8d70d26d194b1e8d1ccb54e7695bf0" member_type: VOTER last_known_addr { host: "127.18.184.188" port: 46131 } } }
I20260808 02:17:39.004377 11610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:39.009315 11672 mvcc.cc:205] Tried to move back new op lower bound from 7316092759907074048 to 7316092759720706048. Current Snapshot: MvccSnapshot[applied={T|T < 7316092759907074048}]
I20260808 02:17:39.024361 11665 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:39.025710 11665 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:39.034843 11665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4e24a82498e40e5b7866ab09c5f2d29: Generated new TSK 0
I20260808 02:17:39.036114 11665 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:17:39.060516 11608 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.058729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60290 (local address 127.18.184.190:35069)
0808 02:17:39.058876 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.058882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.059175 (+ 293us) server_negotiation.cc:407] Connection header received
0808 02:17:39.059421 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.059425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.059486 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.059552 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.060360 (+ 808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60290: BlockingRecv error: recv got EOF from 127.0.0.1:60290 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 02:17:39.060635 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.058815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.18.184.189:34895)
0808 02:17:39.058938 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.058942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.059596 (+ 654us) server_negotiation.cc:407] Connection header received
0808 02:17:39.059793 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.059796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.059857 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.059915 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.060505 (+ 590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60662: BlockingRecv error: recv got EOF from 127.0.0.1:60662 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 02:17:39.060691 11605 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.059211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48088 (local address 127.18.184.188:46131)
0808 02:17:39.059350 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.059356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.059709 (+ 353us) server_negotiation.cc:407] Connection header received
0808 02:17:39.059892 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.059896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.059945 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.059997 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.060558 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48088: BlockingRecv error: recv got EOF from 127.0.0.1:48088 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 02:17:39.064289 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.063168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60664 (local address 127.18.184.189:34895)
0808 02:17:39.063292 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.063296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.063401 (+ 105us) server_negotiation.cc:407] Connection header received
0808 02:17:39.063564 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.063567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.063640 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.063742 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.064174 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60664: BlockingRecv error: recv got EOF from 127.0.0.1:60664 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 02:17:39.068524 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.067647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60670 (local address 127.18.184.189:34895)
0808 02:17:39.067775 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.067778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.067789 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:17:39.067829 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.067831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.067871 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.067932 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.068397 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60670: BlockingRecv error: recv got EOF from 127.0.0.1:60670 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 02:17:39.073625 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.072418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60682 (local address 127.18.184.189:34895)
0808 02:17:39.072542 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.072545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.072768 (+ 223us) server_negotiation.cc:407] Connection header received
0808 02:17:39.072921 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.072925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.072980 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.073064 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.073516 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60682: BlockingRecv error: recv got EOF from 127.0.0.1:60682 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 02:17:39.080989 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.079748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60692 (local address 127.18.184.189:34895)
0808 02:17:39.079905 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.079910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.080133 (+ 223us) server_negotiation.cc:407] Connection header received
0808 02:17:39.080270 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.080272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.080318 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.080385 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.080873 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60692: BlockingRecv error: recv got EOF from 127.0.0.1:60692 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 02:17:39.092659 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.091479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60708 (local address 127.18.184.189:34895)
0808 02:17:39.091637 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.091642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.091731 (+ 89us) server_negotiation.cc:407] Connection header received
0808 02:17:39.091842 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.091846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.091909 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.092043 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.092531 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60708: BlockingRecv error: recv got EOF from 127.0.0.1:60708 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 02:17:39.112727 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.111046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.18.184.189:34895)
0808 02:17:39.111250 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.111256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.111611 (+ 355us) server_negotiation.cc:407] Connection header received
0808 02:17:39.111854 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.111858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.111922 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.112002 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.112573 (+ 571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60710: BlockingRecv error: recv got EOF from 127.0.0.1:60710 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 02:17:39.148654 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.147279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60726 (local address 127.18.184.189:34895)
0808 02:17:39.147484 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.147489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.147652 (+ 163us) server_negotiation.cc:407] Connection header received
0808 02:17:39.147844 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.147852 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.147922 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.148014 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.148521 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60726: BlockingRecv error: recv got EOF from 127.0.0.1:60726 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260808 02:17:39.215919 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.214839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60738 (local address 127.18.184.189:34895)
0808 02:17:39.214974 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.214978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.215085 (+ 107us) server_negotiation.cc:407] Connection header received
0808 02:17:39.215233 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.215236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.215284 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.215391 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.215817 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60738: BlockingRecv error: recv got EOF from 127.0.0.1:60738 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 02:17:39.347906 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.346515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60750 (local address 127.18.184.189:34895)
0808 02:17:39.346671 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.346679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.346820 (+ 141us) server_negotiation.cc:407] Connection header received
0808 02:17:39.346954 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.346956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.347011 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.347121 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.347752 (+ 631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60750: BlockingRecv error: recv got EOF from 127.0.0.1:60750 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 02:17:39.607365 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:39.606080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60760 (local address 127.18.184.189:34895)
0808 02:17:39.606230 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:39.606236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:39.606404 (+ 168us) server_negotiation.cc:407] Connection header received
0808 02:17:39.606579 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:39.606581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:39.606633 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:39.606722 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:39.607232 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60760: BlockingRecv error: recv got EOF from 127.0.0.1:60760 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:17:39.771610 11623 catalog_manager.cc:1259] Loaded cluster ID: 64fe63929fde4eb898bd4b1d74d28e12
I20260808 02:17:39.771703 11623 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf: loading cluster ID for follower catalog manager: success
I20260808 02:17:39.773497 11623 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf: acquiring CA information for follower catalog manager: success
I20260808 02:17:39.774705 11623 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0832c5eea1b24f1eb656c019a0e134bf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:17:39.800311 11641 catalog_manager.cc:1259] Loaded cluster ID: 64fe63929fde4eb898bd4b1d74d28e12
I20260808 02:17:39.800400 11641 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0: loading cluster ID for follower catalog manager: success
I20260808 02:17:39.802090 11641 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0: acquiring CA information for follower catalog manager: success
I20260808 02:17:39.803323 11641 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3f8d70d26d194b1e8d1ccb54e7695bf0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:17:40.126430 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:40.121832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60774 (local address 127.18.184.189:34895)
0808 02:17:40.122165 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:40.122169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:40.122196 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:17:40.122380 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:40.122383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:40.122530 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:40.122751 (+ 221us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:40.123013 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:40.123715 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:40.124750 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:40.125024 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:40.125117 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:40.125517 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:40.126001 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:40.126102 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:40.126169 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 02:17:40.126226 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":89,"threads_started":1}
I20260808 02:17:40.134464 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:40.130078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48096 (local address 127.18.184.188:46131)
0808 02:17:40.130387 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:40.130391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:40.130409 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:17:40.130509 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:40.130511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:40.130725 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:40.130855 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:40.131084 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:40.131739 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:40.132683 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:40.132938 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:40.133043 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:40.133402 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:40.133953 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:40.134068 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:40.134153 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 02:17:40.134212 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":110,"threads_started":1}
I20260808 02:17:40.142392 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:40.137997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60788 (local address 127.18.184.189:34895)
0808 02:17:40.138209 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:40.138214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:40.138491 (+ 277us) server_negotiation.cc:407] Connection header received
0808 02:17:40.138745 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:40.138749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:40.138820 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:40.138886 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:40.139158 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:40.139843 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:40.141005 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:40.141401 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:40.141514 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:40.141810 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:40.141899 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:40.141947 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:40.142121 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 02:17:40.142190 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260808 02:17:40.142637 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:40.138424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.18.184.188:46131)
0808 02:17:40.138545 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:40.138549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:40.139109 (+ 560us) server_negotiation.cc:407] Connection header received
0808 02:17:40.139162 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:40.139166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:40.139238 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:40.139314 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:40.139566 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:40.140315 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:40.141301 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:40.141633 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:40.141738 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:40.142010 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:40.142100 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:40.142165 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:40.142470 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 02:17:40.142526 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:17:40.143461 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:40.137958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60300 (local address 127.18.184.190:35069)
0808 02:17:40.138240 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:40.138245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:40.138455 (+ 210us) server_negotiation.cc:407] Connection header received
0808 02:17:40.138728 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:40.138736 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:40.138860 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:40.139006 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:40.139271 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:40.140120 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:40.141247 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:40.141628 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:40.141738 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:40.142358 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:40.142747 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:40.142821 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:40.143215 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 02:17:40.143292 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":69,"threads_started":1}
I20260808 02:17:40.146622 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 11361
I20260808 02:17:40.156334 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 11442
I20260808 02:17:40.164992 19170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmhslVu/build/debug/bin/kudu with pid 11526
2026-08-08T02:17:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3386 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6685 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (298320 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST